Estimation of Outsourced Projects
- Posted on June 6, 2007 7:59:AM by Steve McConnell to 10x Software Development
- Technique, estimation, Management
A question we sometimes hear from our clients is, "My company does outsource software development for other companies. Is there anything special about estimating in that context?" There actually are some distinctive aspects to estimating in the context of preparing a bid or price quote, and I don't discuss that in my book Software Estimation.
Estimation in a Time & Materials Context
Creating estimates to support time and materials bids (i.e., charging by the hour), is only barely a special case because the very structure of T&M implies some variability in the outcome, same as my recommendations for estimating in-house development work. The only real difference if you can even call it a difference is that you have to make doubly sure that you're setting expectations clearly: "This is an estimate. We can't know the outcome with 100% certainty. Actual results will depend on exact details of what you end up requiring and how different issues get prioritized throughout the project," and so on.
Estimation in a Fixed-Price Context
In contrast, estimation in a fixed-price context is very much a special case. If your estimate causes you to bid too high, you won't get the work. If it causes you to bid too low, you will lose money. Both of these are undesirable outcomes! In other circumstances I usually find myself recommending that people back away from really elaborate estimation approaches because there's so much inherent variability in software projects that the accuracy of your estimates is inherently limited, and you reach the point of diminishing estimation accuracy after you've put in even a little bit of effort. But a fixed price environment, at proposal time, is one of the few circumstances I've run encountered in which an elaborate estimation approach is warranted. And so my first recommendation is, If your business depends on creating fixed price bids, focus on estimation skills as a core competency and treat estimation work as a business-critical function. That means, read Software Estimation, take my company's estimation class, and read other people's estimation books.
My second recommendation is similar to my general recommendation that you separate the "estimate" from the "target." In a fixed price bid context, separate estimation from pricing. The estimate informs the price you'll charge, of course, but there isn't any necessary relationship between the two. You can price a bid at the "unlikely" end of the estimation range if it's really important to you to win the work, and you're willing to lose money on it. Or you can price it way above the estimation range if you think you have an approach that allows you to perform the work at low cost to you and that delivers a higher value to the client.
We've seen lots of companies wrap themselves around the axle when the sales staff insists on lowering the "estimate" to get the work, when really what needs to be lowered is the price. This creates confusion throughout the project. Giving everyone permission to keep estimates and prices separate increases accountability on the sales side (they have to own up to the fact that they're pricing something on the low end of the estimation range and get buy in to do that), and it improves planning on the dev side -- if there's a big gap between the price and the estimate, the project needs to be treated as a higher risk project than if there isn't a large gap. When estimation and pricing are merged into one concept and called "estimation" (even though it isn't really estimation), the project planners can lose the important risk information that arises from the relationship between the price and the estimates.
Try not to do the "commitment/pricing" estimate until later in the cone of uncertainty. Of course this is the holy grail, but most companies can't do this with any regularity because they feel that the competitive pressures require them to submit bids in the wider part of the cone.
Bid smaller amounts of work when you can, i.e., be more iterative. One of the great benefits of iterative development is the ability to generate project-level data on early iterations that can be used to estimate later iterations with really good accuracy. The companies we've worked with have settled in on 3 iterations as the number needed to calibrate a project team's productivity. Interestingly enough, it doesn't seem to matter whether the iterations are 1 week or 1 month or longer -- it still takes 3 iterations.
Consider creating two-phase bids when you can. You can call the first phase "preliminary work", "exploratory phase," "proof of concept phase," "design phase", "Phase 1", etc. The purpose of this phase is to attack all the sources of high variability feeding into the estimates and ultimately deliver a bid for the second part of the project after the cone has been narrowed considerably. We've seen many companies use this approach successfully, although I can't think of any companies we've seen that have been able to use it for the majority of the work they bid on. Again, competitive pressures seem to lead to their using this approach only selectively.
Two phase bids can be structured either more "waterfallish" or more "agile". The description above assumes a more linear development approach in which you're trying to get most or all of the requirements defined up front and then bid the whole project. In a more agile approach, you can treat "Phase 1" as an actual design-build-deliver cycle, but structure it into 3 iterations so that you can get good project-level calibration data that you can then use as the basis for bidding the remainder of the project.
Collect historical data on your estimates at proposal time vs. the eventual outcomes so that you can build your own cone of uncertainty. The better records you keep about what materials fed into your estimate, the more meaningful your cone will be. For example, you might have really specific requirements for one bid and pretty vague requirements for another. In one sense, if they're both "proposal time" estimates, you might treat them similarly. But if one was supported by significantly more detail in the requirements that implies you're at a different location in the cone, and you'd want to account for that.
On this particular topic, several of the most powerful recommendations aren't specifically about estimation; they're about project control.
Go highly iterative as early as you can, regardless of whether the bid is structured into one or two phases. Even if you're working to a single-stage bid, there's value to getting project-level calibration data sooner rather than later. If you discover 10% of the way into the project that you've under bid it by a factor of 2, you can go back to the customer sooner and reset their expectations, you can give the customer options that you still have time to act on, you can implement functionality in strict priority order, you can identify the project as a high risk project and manage it accordingly, etc. But if you don't have the project level data that tells you your initial estimates were way off, you'll just run the project as "business as usual", which is really the last thing you want to do.
Document assumptions at the contract level, spell them out in as much detail as you can, and then *contain* them. If you build a house, your building contractor might let you specify the kitchen cabinets, but there will be a line item in the contract budget for cabinets. If you end up choosing cabinets that are more expensive, you pay the difference. You typically would have line items for all kinds of things: lighting, landscaping, carpet, flooring, countertops, etc. The areas that are more certain (e.g., roofing, siding, foundation, plumbing) are simply specified. In software projects we also typically have areas that we can specify in detail and other areas that we don't know enough about at contract time to specify in detail. So in software contracts you can include clauses like, "The exact work required in the XYZ module has been budgeted at 40 staff hours. If work on XYZ exceeds its budget, the contract price will be increased correspondingly." I'm not an attorney so I am not recommending this as specific contract language, but hopefully this gives you a general idea about the general kind of clause you would ask your attorney to include in a contract.
Manage your set of projects/bids as an investment portfolio, accepting that some will "win" and some will "lose." From a theoretical point of view, if you're estimating early in the cone there just isn't a good answer to improving the accuracy of your estimates on a project-by-project basis. The fact is, your estimates will be off to varying degrees, and when you happen to get one that's pretty accurate it will be a matter of luck, not skill, because of the inherent limits of the Cone. On the other hand, assuming there isn't any bias in the early-in-the-cone estimates (which can be a huge assumption), you can essentially punt on the question of project-by-project profitability and instead focus on portfolio level profitability. Solving the problem of estimating accurately in the wide part of the cone for an individual project isn't even theoretically solvable. But solving the problem of estimating a collection of projects in the wide part of the cone IS solveable. The key to solving that problem is rooting out any systemic bias in those estimates so that the error tendency is neutral. Then with that set of neutral estimates you simply increase each estimate by the amount you'd like your profit margin to be. If you want it to be 10%, you bid 10% higher than your neutral estimate. This will result in your actual project cost coming in higher than some of your estimates and lower than others, but on balance, assuming no systemic bias, you should make a 10% profit on your portfolio of projects.
Of course this requires that you have several projects in your portfolio, and that there aren't just one or two huge projects whose estimation errors could drown out whatever error was contributed by the smaller projects, and that you can afford to take a loss on some percentage of your projects. And those are big assumptions that might not be true in your specific case.
Bottom Line on Estimating in a Fixed-Price Bidding Context
The bottom line on this particular question is that it isn't possible to solve this particular problem purely using estimation practices. You have to change when you're estimating (later in the Cone), or what you're estimating (e.g., portfolios vs. individual projects), or how many times you estimate (e.g., two-phase bids). And project-control responses (as opposed to estimation responses) and even contract-level responses will probably turn out to be at least as useful as estimation responses.