Estimation Reading List

Primary References

McConnell, Steve. Software Estimation: Demystifying the Black Art, Redmond, Wa.: Microsoft Press, 2006. The companion book to this class, focuses on art-based estimation techniques. Full details are available at www.stevemcconnell.com/est.htm.  

Stutzke, Richard D. Estimating Software-Intensive Systems, Upper Saddle River, N.J.: Addison Wesley, 2005. Stutzke provides several additional methods for estimating schedule, most of which are more mathematically intensive than the techniques described in this class. Chapter 5 of Stutzke’s book discusses judgment-based estimation techniques and provides background on some of the math described in this chapter. Chapters 8 and 9 describe additional size estimation techniques including Use Case Points, Application Points, Web Objects, and simplified function point techniques. Stutzke also discusses size estimation on COTS (commercial off the shelf) projects. Chapter 12 describes approaches to effort allocation that are based on Cocomo 81 and Cocomo II. Chapters 15 and 23 of Stutzke’s book focuses on detailed cost estimation issues. Cost estimation and other cost-related issues are a major focus of Stutzke’s book, and various cost-related tips are sprinkled throughout. Sections 12.1 and 12.2 discuss the relationships between effort, duration, and staff availability. Chapter 20 describes how to create a WBS. Appendix C of Stutzke’s book contains a summary of measures of estimation accuracy.

Tockey, Steve. Return on Software, Boston, Mass.: Addison Wesley, 2005. Chapter 15 of Tockey’s book contains a good discussion of determining unit cost, including methods of allocating overhead using different costing methods and hazards associated with some of the methods.Chapters 21-23 discuss basic estimation concepts, general estimation techniques, and allowing for inaccuracy in estimates. Tockey includes a detailed discussion of how to build your own cone of uncertainty. Chapter 22 of Tockey’s book discusses estimation with multiple methods. Chapter 23 discusses how to account for inaccuracy in estimates, and Chapters 24-25 discuss how to make decisions under risk and uncertainty.

Secondary References

Armstrong, J. Scott, ed., 2001. Principles of forecasting: A handbook for researchers and practitioners. Boston: Kluwer Academic Publishers, 2001. Armstrong is one of the leading researchers in forecasting in a marketing context. Many of the observations in this book are relevant to software estimation. Armstrong has been a leading critic of overly complex estimation models.

Boehm, Barry W. Software Engineering Economics. Englewood Cliffs, New Jersey: Prentice-Hall, Inc., 1981. Although this edition has been largely superseded by Software Cost Estimation with Cocomo II (below), this edition contains interesting, detailed reference tables for effort and schedule breakdown across activities. Chapter 21 describes a Seven step approach to estimating software projects. Section 22.2 of Boehm’s book describes the original Delphi method and Boehm’s creation of Wide Band Delphi.

Boehm, Barry, et al. Software Cost Estimation with Cocomo II, Reading, Mass.: Addison Wesley, 2000. This book is the definitive description of Cocomo II. The book’s size is daunting, but it describes the basic Cocomo model within the first 80 pages, including detailed definitions of the effort multipliers and scaling factors discussed in this chapter and how Cocomo II accounts for diseconomies of scale. The rest of the book describes extensions of the model. Boehm was the first to popularize the Cone of Uncertainty (he calls it a funnel curve). This book contains his most current description of the phenomenon. Appendix A of Boehm’s book describes effort and schedule breakdowns for waterfall projects, MBASE projects, and Rational Unified Process projects. Table A.10 (which is actually six tables) provides detailed breakdowns of effort and schedule across different activities. Appendix E of Boehm’s book contains a checklist that’s useful for precisely defining what constitutes a “line of code.”

Cohn, Mike. Agile Estimating and Planning, Upper Saddle River, N.J.: Prentice Hall Professional Technical Reference, 2006. Cohn’s book contains an extended discussion of story points, including planning considerations as well as estimation techniques. Chapter 5 of Cohen’s book contains a nice description of the difference between ideal effort and planned effort.

Conte, S. D., H. E. Dunsmore, and V. Y. Shen. Software Engineering Metrics and Models. Menlo Park, California: Benjamin/Cummings, 1986. Conte, Dunsmore, and Shen’s book contains the definitive discussion of evaluating estimation models. It discusses the “within 25% of actual 75% of the time” criteria, as well as many other evaluation criteria.

DeMarco, Tom and Timothy Lister. Waltzing with Bears: Managing Risk on Software Projects, New York: Dorset House, 2003. This book presents a readable introduction to software risk management and introduces the “nano-probability” term.

DeMarco, Tom. Controlling Software Projects. New York: Yourdon Press, 1982. DeMarco discusses the probabilistic nature of software projects.

Fenton, Norman E. and Shari Lawrence Pfleeger. Software Metrics: A Rigorous and Practical Approach, Boston, Ma.: PWS Publishing Company, 1997. Chapter 10 contains a detailed discussion of estimating software reliability. If you don’t like equations with symbols like a, b, y, f, l, Õ, å, G, and ò, then this is not the book for you because all these symbols show up this chapter.

Fisher, Roger, William Ury, and Bruce Patton. Getting to Yes, 2nd Ed. New York: Penguin Books, 1991. This book lays out the details of the principled negotiation strategy described in this chapter. The book is packed with memorable anecdotes and makes for interesting reading even if you’re not very interested in negotiation.

Garmus, David and David Herron. Function Point Analysis: Measurement Practices for Successful Software Projects. Boston, Mass.: Addison Wesley, 2001. This book describes function point counting and presents some simplified counting techniques.

Gilb, Tom. Principles of Software Engineering Management. Wokingham, England: Addison-Wesley. Section 7.14 of Gilb’s book describes using project-specific data to refine estimates. The description of evolutionary delivery throughout the book is based on the expectation that projects will build feedback loops that allow them to be estimated, planned, and managed in a way that allow the project to be become self-correcting.

Goldratt, Eliyahu M. Critical Chain, Great Barrington, MA: The North River Press, 1997. Goldratt describes an approach to dealing with student syndrome as well as an approach to buffer management that addresses Parkinson’s Law.

Grady, Robert B. and Deborah L. Caswell. Software Metrics: Establishing a Company-Wide Program, Englewood Cliffs, NJ: Prentice Hall, 1987.

Grady, Robert B. Practical Software Metrics for Project Management and Process Improvement. Englewood Cliffs, N.J.: PTR Prentice Hall, 1992. These two books describe Grady’s experience setting up a measurement program at Hewlett-Packard. The books contains many hard-won insights into the pitfalls of setting up a measurement program and some interesting examples of the useful data you can ultimately obtain.

Humphrey, Watts S. A Discipline for Software Engineering. Reading, Mass: Addison Wesley, 1995. Humphrey lays out a detailed methodology by which developers can collect personal productivity data, compare their planned results to their actual results, and improve over time. Chapter 5 of Humphrey’s book discusses proxy-based estimation, which he calls the PROBE method, and goes into detail on some supporting statistical techniques. Chapter 5 also discusses Fuzzy Logic. Appendix A contains a short, readable summary of statistical techniques that are useful for software estimation.

ISBSG. Practical Project Estimation, 2nd Edition: A Toolkit for Estimating Software Development Effort and Duration, Australia: International Software Benchmarking Standards Group, February 2005. This book contains numerous useful formulas for computing effort estimates from size estimates. The book is refreshingly candid about the accuracy of its formulas; it includes sample size and r-squared values you can use to assess the validity of its formulas.

Jones, Capers. Applied Software Measurement: Assuring Productivity and Quality, 2d Ed. New York: McGraw-Hill, 1997. Jones discusses the history of function points in detail and presents the arguments against lines of code measures.  Chapter 3 of this book presents an excellent discussion of the sources of error in size, effort, and quality measurements.

Jones, Capers. Estimating Software Costs, New York: McGraw-Hill, 1998. Chapter 14 of Jones’s book contains a detailed discussion and examples of how cost buildups can vary between different kinds of organizations. Chapter 21 explains how unpaid overtime affects cost estimates.

Jones, Capers. Software Assessments, Benchmarks, and Best Practices, Reading, Mass.: Addison Wesley, 2000. Jones’s book provides some data that is updated or expanded from the data he presents in Estimating Software Costs.

Jørgensen M., “A Review of Studies on Expert Estimation of Software Development Effort,” 2002. This paper presents a comprehensive review of the research on expert estimation approaches. The author draws numerous conclusions from the common research threads and presents 12 tips for achieving accurate expert estimates.

Laranjeira, Luiz. “Software Size Estimation of Object-Oriented Systems,” IEEE Transactions on Software Engineering, May 1990. This paper provided a theoretical research foundation for the empirical observation of the Cone of Uncertainty.

Larsen, Richard J. and Morris L. Marx. An Introduction to Mathematical Statistics and Its Applications, 3rd Ed., Upper Saddle River, N.J.: Prentice Hall, 2001. This book is a fairly readable introduction to mathematical statistics; at least it’s readable when you consider the subject matter. It’s an unavoidable fact that if you want to use statistical techniques sooner or later you’ll have to do some math!

Lorenz, Mark and Jeff Kidd, 1994. Object-Oriented Software Metrics, Upper Saddle River, N.J.: PTR Prentice Hall, 1994. Lorenz and Kidd present numerous suggestions of quantities that can be counted in object-oriented programs.

McGarry, John, et al. Practical Software Measurement: Objective Information for Decision Makers, Boston, Mass.: Addison Wesley, 2002. Section 5.1 discusses considerations to include in an estimation procedure.

NASA SEL. Manager’s Handbook for Software Development, Revision 1. Document number SEL-84-101. Greenbelt, Maryland: Goddard Space Flight Center, NASA, 1990. This document describes the NASA SEL’s estimation approach in more detail.

NASA, “ISD Wideband Delphi Estimation,” Number 580-PROGRAMMER-016-01, September 1, 2004, http://software.gsfc.nasa.gov/AssetsApproved/PA1.2.1.2.pdf. This document describes a Wide Band Delphi technique used by the NASA Goddard Space Flight Center.

Putnam, Lawrence H. and Ware Myers. Five Core Metrics. New York: Dorset House, 2003. This book presents a compelling argument for collecting data on the five core metrics of size, productivity, time, effort, and reliability. Chapter 4 contains an extended discussion of the importance of predictability compared to other project objectives. Chapter 11 goes into detail about the efficiency penalty for exceeding seven people on a medium-sized business systems project.

Putnam, Lawrence H. and Ware Myers. Measures for Excellence: Reliable Software On Time, Within Budget. Englewood Cliffs, N.J.: Yourdon Press, 1992. This book describes Putnam’s estimation method including how it addresses diseconomies of scale. I like Putnam’s model because it contains few control knobs and works best when it is calibrated with historical data. The book is mathematically oriented, so it can be slow going. Putnam and Myers also provide numerous useful rules of thumb for planning. The overall context of Putnam and Myers’ book is a detailed, mathematical explanation of Putnam’s estimation model.

Software Engineering Institute’s Software Engineering Measurement and Analysis (SEMA) website, www.sei.cmu.edu/sema/. This comprehensive website helps organizations create data collection (measurement) practices, as well as practices for using the data they collect.

Wiegers, Karl. “Stop Promising Miracles,” Software Development, February 2000. Wiegers’ paper describes a variation on the Wide Band Delphi technique described in this chapter.

Wiegers, Karl. Software Requirements, 2d Ed. Redmond, WA: Microsoft Press, 2003. Wiegers describes numerous practices that help elicit good requirements in the first place, which substantially reduces requirements volatility later in a project.

www.construx.com/resources/surveyor/. This site provides a free code-counting tool called Construx Surveyor.

www.ifpug.org. The International Function Point User Group is the definitive source for current function point counting rules.

Search for more resources