Chief Programmer Team Update

  1. Posted on March 31, 2008 5:28:PM by Steve McConnell to 10x Software Development
  2. Technique, project management, productivity, Management, Team Organization, Chief Programmer Team, Articles

One spinoff from the 10x difference in programmer productivity was the Chief Programmer Team structure. The idea of the chief-programmer team was originally developed at IBM during the late 1960s (Baker 1972, Baker and Mills 1973). It was popularized by Fred Brooks in the Mythical Man-Month (Brooks 1975, 1995), in which Brooks referred to it as a surgical team. The two terms are interchangeable. I described the technique in my 1996 book Rapid Development, but I think we've learned some important lessons about the CPT structure since then.

Original Chief Programmer Team Project

The original chief programmer team project was conducted in the late 1960s. IBM commissioned to build an information retrieval system for the New York Times. The Chief Programmer on that project (the original Chief Programmer) was Harlan Mills, who created all the design and wrote all of the production code. He had eight other people arrayed around him in various support functions:

  • A "backup programmer" serves as the chief programmer’s alter ego. The backup programmer supports the chief programmer as critic, research assistant, technical contact for outside groups, and backup-up chief.
  • The "administrator" handles administrative matters such as money, people, space, and machines. The Chief has ultimate say about these matters, but the administrator frees the Chief from having to deal with them on a daily basis.
  • The "toolsmith" is responsible for creating custom tools requested by the Chief . In today’s terminology, the toolsmith would be in charge of maintaining the build environment, creating scripts, etc.
  • The team is rounded out by a "language lawyer" who supports the Chief by answering esoteric questions about the programming language the Chief is using.

Several of the support roles suggested in the original chief-programmer proposal are now regularly performed by nonprogrammers--by documentation specialists, test specialists, and program managers. Other tasks such as word processing and version control have been simplified so much by modern software tools that they no longer need to be performed by support personnel. And the internet has reduced the need for language lawyers--many questions can be answered via a quick search on the web.

Attempts to Replicate IBM’s Chief Programmer Team Results: Is 10x Good Enough?

On the original project, Harlan Mills personally wrote 83,000 lines of production code in one year. He wrote that code on a batch mode operating system. And on punch cards! Even when you divide the 83,000 lines of code by the nine people on the project, that’s 9,200 lines of code per staff year, which is still in the ballpark of acceptable productivity for similar kinds of projects 40 years later. With productivity like that under those circumstances you can see why the IBM project was heralded as one of the most successful projects of its time.

In the years since that project many organizations have tried to implement Chief Programmer teams, and few have been able to repeat IBM’s initial stunning success. The achilles heel of the Chief Programmer Team model is that for it to make sense to organize staff the way they were organized on the IBM project, the Chief Programmer has to be more productive than everyone else on the team put together. On the original IBM project, Harlan Mills was a near-genius programmer who was an expert software methodologist, talented writer, exceptionally self-disciplined, and highly motivated. When he decided to roll up his sleeves and write code, he had few peers. Think "Kent Beck of His Day" and you’d be pretty close. He was one of the rare individuals truly capable of doing more work than the eight other people on his team put together.

In a previous blog posting I discussed the fact that numerous studies have found 10-fold variations in productivity between the best and worst programmers with similar levels of experience. For the CPT model to work, the Chief Programmer doesn’t have to be 10x as productive as the worst programmer. He has to do the work of eight or nine people put together, which means he has to be 10x as productive as the average programmer, not 10x as productive as the worst. That’s a very tall order. If you assume the best programmer is 10x as productive as the worst, then the best will be only something like 2-3 as productive as the average programmer, and that isn’t good enough for the CPT model to pay off with a total project team of nine people.

Another factor is that, while numerous studies have found 10x differences among individuals, researchers have not found 10-fold differences among programmers working within the same organizations. Some research has found that good programmers tend to cluster within certain companies, average programmers tend to cluster within other companies, and so on (Mills 1983). So even if there’s a 10x difference industrywide, the difference you’d typically see within a given company is more like 3-5x from best to worst, which means the difference from best to average is more like 1.5x or 2x within any given company.

Bottom line: The Chief Programmer Team organization can make sense in the rare case in which you have a near genius on your staff--one that is dramatically more productive than the average programmer on your staff. But from I've seen there are far fewer near geniuses than there are near genius wannabees, and that limits the applicability of the technique.

Resources

References

Brooks, Frederick P., Jr. The Mythical Man-Month, Reading Massachusetts: Addison-Wesley, 1975.

Brooks, Frederick P., Jr. The Mythical Man-Month, Anniversary Edition, Reading Massachusetts: Addison-Wesley, 1995.

Baker, F. Terry. "Chief Programmer Team Management of Production Programming," IBM Systems Journal, vol. 11, no. 1, 1972, pp. 56-73.

Baker, F. Terry and Harlan D. Mills. "Chief Programmer Teams." Datamation, Volume 19, Number 12 (December 1973), pp. 58-61.

McConnell, Steve. Rapid Development. Microsoft Press, 1996.

Mills, Harlan D. Software Productivity, Boston, Massachusetts: Little, Brown, 1983.

Adam said:

March 31, 2008 11:35:PM

As a lowly programmer, it sounds like it would be very nice to have this setup at times. Unfortunately I highly doubt that I would be fit for the Chief Programmer position.

How about a setup where there are two Chief Programmers sharing a supporting team? In that case the Chief Programmers would only have to be 1.5-2.5x as productive as the rest of the team--assuming the equation is that simple.

Rasmus said:

April 1, 2008 12:39:AM

I would also think, it could be possible for the "support staff" to be shared between two Chief Programmers. Especially because the need for some of the support functions has decreased with the internet and development of modern version control software, configuration management software etc.

This is just guessing, but perhaps it's even possible for the team to be shared among three CPs - chances are after all that these CPs have somewhat similar impediments along the way during the course of the project.

John said:

April 1, 2008 8:26:AM

Not sure about your math here. Wasn't the point of the mythical man month that you can't just add people to gain productivity? 10 programmers who have a factor 1 productivity will be less productive than one programmer with a factor 10 productivity since the bureaucracy involved in coding grows non-linearly with team size. So your Chief Programmer might only need to be 3x as good as your average programmer for the surgical team approach to make sense.

Steve McConnell said:

April 1, 2008 10:50:AM

> So your Chief Programmer might only need to be 3x as
> good as your average programmer for the surgical team approach to make sense.

You're right that 10 people who have a factor 1 productivity won't collectively produce a factor of 10 productivity. There is some diseconomy of scale that occurs, but the effect at the relatively small team sizes we're discussing would be to reduce total output to 7x or 8x, not to anything close to 3x.

So the Chief Programmer really does need to be as productive as 7-8 average programmers to break even -- and more like 8x - 10x for the CPT structure to be an advantage.

Steve McConnell said:

April 1, 2008 10:52:AM

> How about a setup where there are two Chief Programmers sharing a supporting team?

In fact, I think this is the direction that the industry has gone. We have program managers, buildmasters, testers, test leads, dev leads, architects, centralized tool groups, and so on. But if you end up with a ratio of 1 programmer to 1 or 2 people in other specializations, that isn't the CPT--that's just staff specialization.

Daniel Yokomizo said:

April 1, 2008 10:57:AM

Aside: I spent twenty minutes writing down a detailed comment, based on studies I did in a company I worked for but (as your commenting software requires javascript, doesn't degrade gracefully and I browse with javascript disabled as default) my comment was lost when I tried to add it. I'll do a summary of my comment without details and ask you to please try to make the comment process work better without javascript enabled.

The best programmer don't need to be 10x more productive than the average programmer, because the average include the better programmer's productivity. We need to compare against the average productivity of the rest of the team, or against the median if we assume that the worse programmers aren't negative (which IME is true, their work have to be tested and rewritten later by better programmers). In a study I did the best programmer was 3x more productive than the average team productivity, 10x than the worst programmer in the team and 6x than the rest of the team (i.e. the average excluding him). Also as he did infrastructure that improved the team's productivity (the productivity was measure against his delivering features time, not the infrasctructure programming time, which was kept separate in the time logs) by 2x (when compared with same-sized teams working at the same time for the same customer following the same development process and the same technology and business complexity) that skews the measures a little because the actual productivity for the rest of the team would be lower than the measured if the common infrastructure wasn't built by the better programmer.

Also I agree with John, when you add programmers the communication requires superlinear effort. In another study I did the best programmers suffered from this superlinear degradation of productivity for teams above size 5, the median programmers suffered a linear degradation and the worse programmers kept the same.

Steve McConnell said:

April 1, 2008 11:54:AM

Daniel, Interesting comment. Based on your description of your data, with the best programmer 10x as productive as the worst and 6x as productive as the average without him, that suggests that the average programmer's productivity is closer to the worst programmer's productivity than it is to the best or even to the midpoint between the best and the worst. (Another way of putting this is that programmer productivity in your organization doesn't follow a symmetric bell curve distribution -- it follows a distribution that is sharply skewed to the left.) Is that correct?

If so, it sounds like you have one of those rare cases in which the best programmers in your organization truly are so much better than the average that you could afford to have 4-5 non-best programmers supporting each best programmer.

[P.S. Sorry about the editing problems. If it makes you feel any better, I lost the entire first version of this post for the same reason.]

Daniel Yokomizo said:

April 2, 2008 1:47:PM

@Steve: Yes, I found that pretty unexpected when I saw the data, I even spent a few days reviewing all the time logs to ensure that I didn't commit any mistakes. What I saw was that the median programmers' productivity was very close to the worst programmers' productivity when left without some sort of coaching going on, but when a coach was available (the best programmer acted like a coach to the team) they responded very well, increasing productivity by almost 2x. OTOH I'm afraid that the worst programmer's productivity data is skewed because the manager considered his work completed even if it was full of bugs and required a rewrite by the rest of the team, also this rewrite time was accounted as part of their regular activities so I couldn't correct this problem, probably their productivity was higher than reported by at least 20%.

After a couple of months observing the teams work I concluded that the difference between the worst programmers and the median ones was how they responded to incentives and improvements in the development process. Eventually the company got rid of all worst programmers and management focused on improving the development process to support the median programmers better. It was a really nice success story.

IRT the curve distribution: the best programmer was really a unique case, the next best programmer was almost 40% less productive, but ignoring the unique point the curve was close to a gaussian. If we assume a productivity of 1 as average we had an one sigma being 2x (or .5x), in this measure the best programmer was in 2.6 sigma to the right and the worse programmer 1 sigma to the left (but IMO it should be 2 sigma to the left considering the rework necessary and the amount of bugs he left).

To isolate the best programmer's influence we did later three projects with similar sized teams with another great programmer acting as lead developer and coach to a few median ones. In this case the difference in productivity of the programmers before the project was the better being 5x more productive than the others. At the end of every project the better programmer reduced his productivity to .8x but the rest of the team improved by 2x, effectively making the gap between them be just 2x (it was very funny to find such nice round numbers in real data). A couple of worst programmers were introduced to the teams and the result was similar to the original project: their productivity didn't increase under the coaching, their code had to be rewritten and the gap between the best programmer and the worst was 6x (the smaller gap was due to the best programmer in the project being different from the earlier one).

My analysis was that the median programmers can perform well on familiar territory but their productivity goes down when they face strange bugs or unknown algorithms, OTOH the lead programmer can help them on those issues, keeping a clear path for them to work.

Looking at my previous response I failed to include my conclusion. In most of the projects the chief programmer team model doesn't work because the 10x (than median) programmers are really rare, which puts me in strong agreement with you. What I wanted to say is that even when we have a 10x better programmer, it's better to use him in a team where he acts as technical lead / coach / infrastructure builder, so he has full control of the quality of the code (he gets to decide if a piece of code is good enough for production or needs to be rewritten) and provides support for the rest of the team to be more productive. Even if we consider that his productivity goes a little downwards the benefits to the team as a whole a too big to ignore. OTOH this approach didn't scale to teams larger than five developers (including the lead) in our experiments but I'm not confident to say that it can scale in other organizations.

Steve McConnell said:

April 2, 2008 3:59:PM

Daniel -- your conclusion sounds like a great takeaway to me: if you have someone who's chief programmer material, you get more mileage by putting him in a lead & coaching role than in a chief programmer role.

I also like the taxonomy of programmers: 10x programmers, average programmers (defined as programmers who are capable of improving), and bad programmers (defined as programmers who don't improve even with significant coaching). That all seems right to me (i.e., matches my experience).

Susan said:

April 2, 2008 5:56:PM

I agree with Daniel Yokomizo, that using a 10x programmer as mentor can positively impact the productivity of an entire team. I think part of being a 10x programmer is some innate ability to get the best from other people, as well as oneself. The only people immune to the positive influence of a real 10x'er are the worst programmers. But they seem to be immune to improvement of any kind.

John Rutter said:

April 4, 2008 6:33:AM

Daniel's conclusion does hit on the right point - creating good quality output from a great team can mean using the *best* programmer to bring on the others, not to do the code directly.

That should result in better team practices, removing the dependency on that one individual (should they move to another project, or leave the company). A good trade-off that allows for increasing scale by having teams that can do the job, not a bunch of individuals with no way to spread the load.

Think this is the point I tried to make in my post on the 10x thread before, but the posters here have made it much clearer.

Thomas Guest said:

April 4, 2008 6:34:AM

An good read, thanks. I'm interested to see a "language lawyer" on the team and wondered if this was a term used originally by IBM in the team you describe, or whether it's your own term for this role? In either case, it's interesting that the chief programmer doesn't necessarily know the programming language inside-out, and relies on having an expert assistant for this.

Paul W. Homer said:

April 4, 2008 10:03:AM

I did five years as essentially the Chief Programmer for a enterprise-class software product in a tiny company. We were highly iterative, with very short implementations (1 to 3 months), and all the members on the team were specialized. I did most of the code (80%), I had a graphic designer, a couple of people building system templates, a writer (sometimes) and someone handling the configuration. Occasionally I'd add a second or third programmer, but the 'projects' were always very targeted. This arrangement worked extremely well.

The thing is, as much as I want to see myself as a "near-genius programmer who was an expert software methodologist, talented writer, exceptionally self-disciplined, and highly motivated", I probably am not that distinct. I code confidently, well and I don't crack under pressure. But it was not necessarily 'who' I was, as more of the way I was going about it.

I think that we should be able to get similar results with bigger teams, so long as the organization doesn't break down into committees. Granted you'll always need to center these groups around 'good programmers', but I don't think they are only dependent on 'great' ones.

Lest I hope not :-)

Paul.

theprogrammersparadox.blogspot.com

Steve McConnell said:

April 8, 2008 9:06:AM

@Thomas - I'm on vacation and don't have the original articles with me -- my best recollection is that language lawyer was the term used in the original article. In any case, the *role* of someone who gets into the nitty gritty of the language so that the Chief Programmer doesn't need to was definitely a role on the original CPT.

Now that you point it out, I do find it interesting that the CP isn't a language lawyer himself. Most of the best programmers I've worked with have prided themselves on their detailed knowledge of language minutea. On the other hand, it's hard to know what exactly was needed to be a "language lawyer" in 1968 vs. today. With the internet and ISO standard languages, I would speculate that that information is more readily available today than it was 40 years ago.

Post a Comment:

 
 

Steve McConnell

Steve McConnell is CEO and Chief Software Engineer at Construx Software where he consults to a broad range of industries, teaches seminars, and oversees Construx’s software development practices. In 1998, readers of Software Development magazine named Steve one of the three most influential people in the software industry along with Bill Gates and Linus Torvalds.

Steve is the author of Software Estimation: Demystifying the Black Art (2006), Code Complete (1993, 2004), Rapid Development (1996), Software Project Survival Guide (1998), and Professional Software Development (2004). His books twice won Software Development magazine's Jolt Excellence award for outstanding software development book of the year.

Steve has served as Editor in Chief of IEEE Software magazine, on the Panel of Experts of the SWEBOK project, and as Chair of the IEEE Computer Society’s Professional Practices Committee.

Steve received a Bachelor’s degree from Whitman College, graduating Magna Cum Laude, Phi Beta Kappa, and earned a Master’s degree in software engineering from Seattle University.
Contact Steve