Classic Build Maturity path

Often there are some questions from the management with respect to build management space, especially when code promotions/release take more time, late night fire fighting over a period or build promotion flunked and QA team raises concerns

1. What is my build maturity?

2. Where does it stand against industry best practices?

3. My build engineer says we have the best process but still it fails often, takes longer duration and I feel some gap….

I tried putting together a classic build maturity path which could help us to comprehend the current state and best state too…I certainly think the maturity could vary based on the application/product architecture and there could be genuine reasons towards some of the current facts … however, I trust this could be a starting point when we stand clueless

I categorize them from Stage 0 thru Stage 4 as shown below

If you are following Agile methodology, I swear anything lesser than Stage 4 is affecting your delivery, quality & productivity. There are plenty of tools in the market which could help you to uplift from wherever you are today and reach Stage 4 fairly in about 6 months time.

Hope this post is useful.


Find and replace a content in Web.Config using MSBuild….

If you wonder how to read & update a specific node in web.config(or any other xml file) using MsBuild, here is a sample…

This is very handy for automatic, unattended deployments…

You need to have Tigris MSBuild Community tasks installed ….. this script uses XMLRead & XMLUpdate tasks from that MSBuild extension….

You need to have .NET FWK 3.5 installed too…



xmlns=”; ToolsVersion =”3.5″ InitialTargets=”FindReplace”>
<!– Required Import to use MSBuild Community Tasks –>
<UsingTask AssemblyFile=”$(MSBuildExtensionsPath)\MSBuildCommunityTasks\MSBuild.Community.Tasks.dll”
<UsingTask AssemblyFile=”$(MSBuildExtensionsPath)\MSBuildCommunityTasks\MSBuild.Community.Tasks.dll”
<!– *********************  Target to read the xml file ********************************–>
<Target Name=”Read”>
<!– Read Test Service end point –>
<Output TaskParameter=”Value” PropertyName=”TestServiceEndPoint” />
<Message Text=”$(TestServiceEndPoint)”/>
<!– Read ConnectionString –>
<Output TaskParameter=”Value” PropertyName=”ConnectionString” />
<Message Text=”(ConnectionString)”/>
<!– *********************  Target to find and replace a value within the xml file *************–>
<Target Name =”FindReplace”>
Value=”CrapCrapCrap123456″ />

Essential build management for modern day software development

Some of the items that we should be watchful inorder to make configuration & build management more productive and useful for the whole SDLC….
here are some common challenges and after effects if we ignore to take care of them….

Repeatability & reliability

-Is about being able to do the same thing over and over
-Is something repeatable after  say 6 months
-Do developers have the ability to pullout a code and assembly which is in live production today

-Does the process produces correct and accurate results every time
-Are you confident that the code is being delivered into the build has the defect,  not the build/package/deploy process itself

Lack of ‘Repeatability & reliability’

Low defect fix rate
-Not able to repeat the build which is in production already..
-Do we have the ability to reconstruct the development environment with the PROD code, fix a defect and ship a patch quickly
-How long does it take to reconstruct matching code and assembly on a developer box? Should be in minutes…
-Inaccurate fixes

Nonstop issues with reliability of build execution
-When a defect found,  are you sure the problem is with the code or could it be the build process
Management lost confidence on build process

Traceability & completeness

-Ways to understand the complete life cycle of  defect/feature that goes into a build….

-Ability to trace it back completely and figure out whether the build contains all of what was intended…
-Does it add any value towards the program goals and objectives
– Why and What are we delivering in this build

Lack of ‘traceability and completeness’

–Not being able to say exactly what is in the build…. 
    ->what new features, enhancement, defects have been added and why?
 –Incomplete builds  & Missing builds 
Missing some artifacts
    ->Post deploy manual hacks/changes on the environments beyond build process
    ->Sometimes builds are missing from the source control
–Not being able to say where the build is being used? 
   ->Where the build has been deployed? Is it being testing on different environments? What version do I have currently on different environments?
–Not being able to say how the build was carried out…. 
  ->Did the source got baselined?
  ->With or without third party assemblies? What version of 3rd party assemblies used?
  ->Was it environment specific build? Any configuration items change based on the environment?
  ->Were there any special compiler/packaging options, instructions followed?

Speed, Agility 

-Is about how quickly a developer can integrate defects and test his changes?
-How fast and integrated is the build process
-Is the process efficient & has absolutely essential steps
-Is it an unattended build/package and deploy process – if manual process is essential, that’s a big bottle neck

-Is about having the build/deploy process in which changes can be integrated
  ->independently, as and when needed

Lack of ‘speed, agility’

Late integration, long builds…. late night firefighting…

–High possibility of incomplete build
– features/enhancements/defects may not meet the entry criteria
–Uncertain defect quality
–Possibility of ‘high defect re-open rates’
–Not being able to integrate changes quickly
-Does the build process take so long – results in weekly build rather than CI builds– what if that build fails
-Travelling with a hidden tiger until next build
–Deferred testing puts the milestone in risk
–Lack of confidence on scheduled build
– not sure what can/can’t go
–Risk of missing milestones in the wake of late integrated testing
–Risk of late night fire fighting towards the end of development cycle – team morale will go down 

So how can the process be improved?

Following high level goals could make you better
–Implement Continuous Integration
•Define a build once in 30-45 mins interval to make sure source code is syntatically correct and produces binary
–Write light weight tools to encourage teams working on Source Control every single minute
•Create some tools to quickly refresh source & assembly based on build number– improve developer productivity
–Envision efficient check in policy & make developer life easy
•like gated checkins to ensure a change is built along with the latest content from the source tree automatically prior to check in
–Automate and make unattended build/package/deploy/sanity testing
•Use best and authentic tools to ensue maximum benefits from automation
–Ensure build process is simple, transparent, fast and easy
•Anyone should be able to initiate and manage the build process with no or negligible training
-Try to adopt Application Life cycle Management model – Integrate tools suite right from Requirements thru testing..

Configuration management metrics….performance indicators

How Integrated Builds, package and deploy process helps to increase overall SDLC productivity…..

Often I see questions, how does good config management help for better productivity or bad config management kills productivity? is there any metrics around that? where do I stand? what should I watch out for?

Here are some common key performance indicators that could help you to review the health of software configuration, build and release management for an integrated development environment.

Problem 1:

High build cycle time….. On an ideal condition for a 900KLOC C# code, complete product compilation can be accomplished in less than 15  minutes with an ordinary build machine….another 20 minutes to package & deploy to a few boxes…..

Problem indicators:

One more level finer, how long does it take you to?

  • Generate a candidate build from development environment? Are you experiencing more and often compilation failure? Especially while integrating the code?
  • Get it packaged according to the product deployment plan?
  • Get those bits deployed on the development environment for initial testing?

Problem 2:

High build rollout time to QA – typically how long does it take you to promote a build to different environments?  a day or more?

Considering candidate build generation is 15 minutes, an efficient build rollout can happen as quickly as 1 hour to any testing environment…..and to as many as 50 tester workstations….

Problem indicators:

  • Where do you get stuck? Environment maintenance? More people cook in your kitchen? do people directly manipulate things on the deployed environment rather than maintaining a source controlled version??
  • Do you manually double click the .msi and install on every box?
  • When you promote the same build to higher environments, should you manually touch up the application configuration parameters according to the environment? Meaning, for dev x=50, for test x=900, how that change is being handled? Manually?
  • Do you manually configure your application?
  • Release coordination consumes more time to get a candidate build to be promoted to the next level? Possibly due to late integration…check problem #1.
  • Have you defined entry and exit criteria for a build?

Problem 3:

How many $$$ do you spend for configuration/build/release management? 100+ hrs for a build rollout? (Possibly because manual build/package/deploy process…. Consumes a lot of time from different roles from release coordinator, build engineer, DBA etc…)
In an ideal world, it need not be more than 2 hrs/per build rollout, may be 5 hrs….

If not…. Here are the problem indicators:

  • Manual build/package/deploy
  • Manual coordination with external teams like DBA…
  • No defined entry/exit criteria..

If your shop is experiencing few or some of the above mentioned challenges…. Certainly there are ways to improve your overall config, build and release management process for an integrated development environment…..get ready to with the following solutions..

Implement Continuous Integration

Many shops still integrate code in an old fashion – say, on a 6 weeks development cycle, 4 weeks independent development (only local builds) and last 2 weeks for integration….typically late integration fakes the progress… lot of hidden tigers will appear when you start to integrate….

If your SCM tool doesn’t support frequent integration, worth considering a switch over to a tool like TFS which supports CI builds out of box….

Do you integrate continuously? NO? are you panic about the frequent changes on your foundational code base..?

Please consider implementing continuous integration – great ways to make sure a code change works well with all others and enables you to test the change immediately in next 30 minutes by efficient build/package/deploy automation…..

Unattended Build/package/deploy process


  • Efficiently automate the build
  • Coach the team in such a way that “build break is a crime” – it should get immediate attention and action…

If it’s manual – that certainly hinder your progress… automate the builds… almost all the tools available in the market have the ability for continuous integration and automated build process…. Make sure compiles assembly is ready for testing at the earliest after the change…..


  • Automate the packaging process
  • Tightly integrate it along with build process – build process should automatically trigger the packaging..
  • Maintain centralized packaging scripts – otherwise you will end up packaging some MS Windows assemblies along with your package.. 😉
  • Keep it simple and fast… say, if .vdproj takes more time to package something… consider trying a PoC with WiX – in my experience WiX has helped us in reducing the size, time and it’s so flexible….

Automated and integrated packaging process can really boost your dev team productivity.
Consider the current packaging model to be manual… post build, the build server sends out an e-mail… build engineer has to manually copy the assemblies to some location and kick start the packaging process. By the way, he/she might accidently forget some assembly while copying or xcopy might simply fail for an assembly which we did not notice…. the .msi generated off of the manual process certainly fakes the testing… and manual process could be more time consuming as well….

Deploy and post deploy:

  • Automate the deploy process – there are plenty of easy tools available like psExec, WinRs, PowerShell scripts etc….
  • Integrate deploy with build/package process…. This is awesome to have… if a developer wants to test a code change….he just have to click a button to Q the build on the build server… then it should build->package->deploy off of 1 central id…this gives another ability for you to restrict all other people from accessing the application servers too… so the environment is so clean….
  • Establish a process to have an “operational application with bare minimum functionality’ at the end of each business day – create another automated build for the end of each day which deploys the latest bits to the environment
  • Establish a build verification test processes
  • If possible automate BVT too… sometimes; BVT might be a killer… daily executing some test cases for hrs might irritate the development community…..

Configuration Management

Finally make some changes to Source Control too…..

  • Envision the source tree structure in such a way that helps rapid development cycle
  • Define efficient check in policies defined to avoid code duplication, confirm unit testing, make sure review process  etc..
  • If your team often breaks builds….consider creating some tiny tools to mimic exactly what happens on the build server…like take the latest & build everything locally in a fashion how that happens on the build server..Otherwise teams might not be working in the source control and often code integration results in failure…..

Largely, continuous integration, integrated builds, package, deploy process can really boost your overall SDLC productivity….