Estimation Reading List
Primary References
McConnell, Steve. Software Estimation: Demystifying
the Black Art, Redmond,
Wa.: Microsoft Press, 2006. The companion book to this class, focuses on
art-based estimation techniques. Full details are available at
www.stevemcconnell.com/est.htm.
Stutzke, Richard D. Estimating Software-Intensive
Systems, Upper Saddle River,
N.J.: Addison Wesley, 2005.
Stutzke provides several additional methods for estimating schedule, most of
which are more mathematically intensive than the techniques described in this
class. Chapter 5 of Stutzke’s book discusses judgment-based estimation
techniques and provides background on some of the math described in this
chapter. Chapters 8 and 9 describe additional size estimation techniques
including Use Case Points, Application Points, Web Objects, and simplified
function point techniques. Stutzke also discusses size estimation on COTS (commercial
off the shelf) projects. Chapter 12 describes approaches to effort allocation
that are based on Cocomo 81 and Cocomo II. Chapters 15 and 23 of Stutzke’s book
focuses on detailed cost estimation issues. Cost estimation and other
cost-related issues are a major focus of Stutzke’s book, and various
cost-related tips are sprinkled throughout. Sections 12.1 and 12.2 discuss the
relationships between effort, duration, and staff availability. Chapter 20
describes how to create a WBS. Appendix C of Stutzke’s book contains a summary
of measures of estimation accuracy.
Tockey, Steve. Return on Software,
Boston, Mass.:
Addison Wesley, 2005. Chapter 15 of Tockey’s book contains a good discussion of
determining unit cost, including methods of allocating overhead using different
costing methods and hazards associated with some of the methods.Chapters 21-23
discuss basic estimation concepts, general estimation techniques, and allowing
for inaccuracy in estimates. Tockey includes a detailed discussion of how to
build your own cone of uncertainty. Chapter 22 of Tockey’s book discusses estimation
with multiple methods. Chapter 23 discusses how to account for inaccuracy in estimates,
and Chapters 24-25 discuss how to make decisions under risk and uncertainty.
Secondary References
Armstrong, J. Scott, ed., 2001. Principles of forecasting: A
handbook for researchers and practitioners. Boston: Kluwer Academic
Publishers, 2001. Armstrong is one of the leading researchers in forecasting in
a marketing context. Many of the observations in this book are relevant to
software estimation. Armstrong has been a leading critic of overly complex
estimation models.
Boehm, Barry W. Software Engineering Economics.
Englewood Cliffs, New Jersey: Prentice-Hall, Inc., 1981.
Although this edition has been largely superseded by Software Cost Estimation with Cocomo II
(below), this edition contains interesting, detailed reference tables for
effort and schedule breakdown across activities. Chapter 21 describes a Seven
step approach to estimating software projects. Section 22.2 of Boehm’s book
describes the original Delphi method and
Boehm’s creation of Wide Band Delphi.
Boehm, Barry, et al. Software Cost Estimation with
Cocomo II, Reading,
Mass.: Addison Wesley, 2000. This
book is the definitive description of Cocomo II. The book’s size is daunting,
but it describes the basic Cocomo model within the first 80 pages, including detailed
definitions of the effort multipliers and scaling factors discussed in this
chapter and how Cocomo II accounts for diseconomies of scale. The rest of the
book describes extensions of the model. Boehm was the first to popularize the
Cone of Uncertainty (he calls it a funnel curve). This book contains his most
current description of the phenomenon. Appendix A of Boehm’s book describes
effort and schedule breakdowns for waterfall projects, MBASE projects, and
Rational Unified Process projects. Table A.10 (which is actually six tables)
provides detailed breakdowns of effort and schedule across different activities.
Appendix E of Boehm’s book contains a checklist that’s useful for precisely defining
what constitutes a “line of code.”
Cohn, Mike. Agile Estimating and Planning,
Upper Saddle River, N.J.: Prentice Hall Professional Technical Reference, 2006.
Cohn’s book contains an extended discussion of story points, including planning
considerations as well as estimation techniques. Chapter 5 of Cohen’s book
contains a nice description of the difference between ideal effort and planned
effort.
Conte, S. D., H. E. Dunsmore, and V. Y. Shen. Software
Engineering Metrics and Models. Menlo
Park, California:
Benjamin/Cummings, 1986. Conte, Dunsmore, and Shen’s book contains the
definitive discussion of evaluating estimation models. It discusses the “within
25% of actual 75% of the time” criteria, as well as many other evaluation criteria.
DeMarco, Tom and Timothy Lister. Waltzing with Bears: Managing
Risk on Software Projects,
New York: Dorset
House, 2003. This book presents a readable introduction to software risk
management and introduces the “nano-probability” term.
DeMarco, Tom. Controlling Software Projects. New York: Yourdon Press,
1982. DeMarco discusses the probabilistic nature of software projects.
Fenton, Norman E. and Shari Lawrence Pfleeger. Software Metrics: A Rigorous and Practical Approach,
Boston, Ma.:
PWS Publishing Company, 1997. Chapter 10 contains a detailed discussion of
estimating software reliability. If you don’t like equations with symbols like a, b, y, f, l, Õ, å, G, and
ò,
then this is not the book for you because all these symbols show up this
chapter.
Fisher, Roger, William Ury, and Bruce Patton. Getting
to Yes, 2nd Ed. New York:
Penguin Books, 1991. This book lays out the details of the principled
negotiation strategy described in this chapter. The book is packed with
memorable anecdotes and makes for interesting reading even if you’re not very
interested in negotiation.
Garmus, David and David Herron. Function Point Analysis: Measurement Practices for Successful Software
Projects. Boston, Mass.: Addison Wesley, 2001. This book
describes function point counting and presents some simplified counting
techniques.
Gilb, Tom. Principles
of Software Engineering Management. Wokingham,
England: Addison-Wesley.
Section 7.14 of Gilb’s book describes using project-specific data to refine
estimates. The description of evolutionary delivery throughout the book is
based on the expectation that projects will build feedback loops that allow
them to be estimated, planned, and managed in a way that allow the project to
be become self-correcting.
Goldratt, Eliyahu M. Critical Chain, Great Barrington, MA: The North River
Press, 1997. Goldratt describes an approach to dealing with student syndrome as
well as an approach to buffer management that addresses Parkinson’s Law.
Grady, Robert B. and Deborah L. Caswell. Software Metrics: Establishing a
Company-Wide Program, Englewood
Cliffs, NJ: Prentice Hall, 1987.
Grady, Robert B. Practical
Software Metrics for Project Management and Process Improvement. Englewood Cliffs, N.J.:
PTR Prentice Hall, 1992. These two books describe Grady’s experience setting up
a measurement program at Hewlett-Packard. The books contains many hard-won
insights into the pitfalls of setting up a measurement program and some
interesting examples of the useful data you can ultimately obtain.
Humphrey, Watts S. A
Discipline for Software Engineering. Reading,
Mass: Addison Wesley, 1995. Humphrey lays out a detailed methodology by which
developers can collect personal productivity data, compare their planned results
to their actual results, and improve over time. Chapter 5 of Humphrey’s book
discusses proxy-based estimation, which he calls the PROBE method, and goes
into detail on some supporting statistical techniques. Chapter 5 also discusses
Fuzzy Logic. Appendix A contains a short, readable summary of statistical
techniques that are useful for software estimation.
ISBSG. Practical Project Estimation, 2nd Edition: A Toolkit for
Estimating Software Development
Effort and Duration, Australia: International Software Benchmarking
Standards Group, February 2005. This book contains numerous useful formulas for
computing effort estimates from size estimates. The book is refreshingly candid
about the accuracy of its formulas; it includes sample size and r-squared
values you can use to assess the validity of its formulas.
Jones, Capers. Applied Software Measurement:
Assuring Productivity and Quality, 2d Ed. New York: McGraw-Hill,
1997. Jones discusses the history of function points in detail and presents the
arguments against lines of code measures.
Chapter 3 of this book presents an excellent discussion of the sources
of error in size, effort, and quality measurements.
Jones, Capers. Estimating Software Costs,
New York:
McGraw-Hill, 1998. Chapter 14 of Jones’s book contains a detailed discussion
and examples of how cost buildups can vary between different kinds of
organizations. Chapter 21 explains how unpaid overtime affects cost estimates.
Jones, Capers. Software Assessments, Benchmarks,
and Best Practices, Reading,
Mass.: Addison Wesley, 2000.
Jones’s book provides some data that is updated or expanded from the data he
presents in Estimating Software Costs.
Jørgensen M., “A Review of Studies on Expert
Estimation of Software Development Effort,” 2002. This paper presents a
comprehensive review of the research on expert estimation approaches. The
author draws numerous conclusions from the common research threads and presents
12 tips for achieving accurate expert estimates.
Laranjeira, Luiz. “Software Size Estimation of Object-Oriented
Systems,” IEEE Transactions on
Software Engineering, May 1990. This paper provided a
theoretical research foundation for the empirical observation of the Cone of
Uncertainty.
Larsen, Richard J. and Morris L. Marx. An
Introduction to Mathematical Statistics and Its
Applications, 3rd Ed., Upper Saddle River, N.J.:
Prentice Hall, 2001. This book is a fairly readable introduction to
mathematical statistics; at least it’s readable when you consider the subject
matter. It’s an unavoidable fact that if you want to use statistical techniques
sooner or later you’ll have to do some math!
Lorenz, Mark and Jeff Kidd, 1994. Object-Oriented Software Metrics, Upper Saddle River, N.J.:
PTR Prentice Hall, 1994. Lorenz and Kidd present numerous suggestions of quantities
that can be counted in object-oriented programs.
McGarry, John, et al. Practical Software Measurement: Objective Information for Decision
Makers, Boston, Mass.: Addison Wesley, 2002. Section 5.1
discusses considerations to include in an estimation procedure.
NASA SEL. Manager’s
Handbook for Software Development, Revision 1. Document number SEL-84-101. Greenbelt, Maryland: Goddard Space Flight
Center, NASA, 1990. This
document describes the NASA SEL’s estimation approach in more detail.
NASA, “ISD Wideband Delphi Estimation,” Number
580-PROGRAMMER-016-01, September 1, 2004, http://software.gsfc.nasa.gov/AssetsApproved/PA1.2.1.2.pdf.
This document describes a Wide Band Delphi technique used by the NASA
Goddard Space Flight Center.
Putnam, Lawrence H. and Ware Myers. Five Core
Metrics. New York: Dorset House, 2003. This book presents a compelling
argument for collecting data on the five core metrics of size, productivity,
time, effort, and reliability. Chapter 4 contains an extended discussion of the
importance of predictability compared to other project objectives. Chapter 11
goes into detail about the efficiency penalty for exceeding seven people on a
medium-sized business systems project.
Putnam, Lawrence H. and Ware Myers. Measures for
Excellence: Reliable Software On Time, Within Budget. Englewood Cliffs, N.J.:
Yourdon Press, 1992. This book describes Putnam’s estimation method including
how it addresses diseconomies of scale. I like Putnam’s model because it
contains few control knobs and works best when it is calibrated with historical
data. The book is mathematically oriented, so it can be slow going. Putnam and
Myers also provide numerous useful rules of thumb for planning. The overall
context of Putnam and Myers’ book is a detailed, mathematical explanation of Putnam’s
estimation model.
Software Engineering Institute’s Software Engineering
Measurement and Analysis (SEMA) website, www.sei.cmu.edu/sema/.
This comprehensive website helps organizations create data collection
(measurement) practices, as well as practices for using the data they collect.
Wiegers, Karl. “Stop Promising Miracles,” Software Development, February 2000.
Wiegers’ paper describes a variation on the Wide Band Delphi technique
described in this chapter.
Wiegers, Karl. Software Requirements,
2d Ed. Redmond, WA: Microsoft Press, 2003. Wiegers describes numerous practices
that help elicit good requirements in the first place, which substantially
reduces requirements volatility later in a project.
www.construx.com/resources/surveyor/.
This site provides a free code-counting tool called Construx Surveyor.
www.ifpug.org. The
International Function Point User Group is the definitive source for current
function point counting rules.
Search for more resources