The largest contributor to optimism bias is the document that starts the project
HM Treasury publishes uplifts by project type, from 24 per cent on a standard building to 200 on equipment and software. The instruction is to start at the upper bound and come down only against verified evidence. The single largest contributory factor it names is not the contractor, the ground or the weather.
There is a table, published by a finance ministry, free to download, that tells you by how much projects of a given type have historically been underestimated.
It has existed since 2002. It is two pages into a document written for exactly the people who prepare sanction papers. And on a great many capital programmes nobody involved has read it.
What the table says
The Supplementary Green Book Guidance on Optimism Bias sets out recommended adjustment ranges for six project types, in two dimensions. The capital expenditure figures are the ones usually quoted.
A standard building carries an upper bound of 24 per cent and a lower bound of 2. A non-standard building, meaning one that is innovative, mostly unique, or complex to construct, runs from 4 to 51. Standard civil engineering, 3 to 44. Non-standard civil engineering, 6 to 66. Equipment and development, which covers software and systems, runs from 10 to 200.
Outsourcing is measured differently, on operating rather than capital expenditure, with an upper bound of 41 per cent.
There is a second table for works duration. A non-standard building carries up to 39 per cent there, standard civil engineering up to 20, equipment and development up to 54.
The instruction almost nobody follows
The numbers are not the interesting part. The method is.
Step two of the guidance says to start with the upper bound. Not the midpoint, not a judgement, not a figure the team is comfortable defending. The upper bound, as the starting value.
You then reduce it, but only in proportion to the contributory factors you can show have been mitigated, using a factor between zero and one. And step five is unambiguous about the evidence required:
Clear and tangible evidence of the mitigation of contributory factors must be observed, and should be independently verified, before reductions in optimism bias are made.
Almost every estimate does the opposite. It begins with a bottom-up build-up, adds a contingency percentage somebody senior finds acceptable, and arrives at a number that is defensible line by line and has never been compared to what projects of that type actually cost.
The guidance also notes something easy to miss. Those upper bounds are the average historic optimism bias observed at outline business case stage. Earlier than that, it says, higher adjustments may be required. The table is not the worst case. It is the middle of the distribution at a point where a fair amount is already known.
What the estimate was declaring about itself
, read: An estimate is a measure of how well the scope is definedThe finding that should change where assurance is spent
Behind Table 1 are three further tables breaking the upper bound down into contributory factors, and they answer a question most organisations never ask: what causes the overrun.
The answer is consistent and it is not what the industry spends its money investigating.
For standard buildings, the largest single contributor to capital expenditure optimism bias is inadequacy of the business case, at 34 per cent of the recorded bias. For non-standard buildings it is again the largest, at 23. For non-standard civil engineering it is the largest again, at 35. For outsourcing, measured on operating expenditure, it accounts for 52.
Not the contractor. Not the ground. Not the weather, the market, or the regulator. The document written before any of them were involved.
The guidance describes what it means by an inadequate business case in plain terms: services that were not anticipated, output specifications that were not defined clearly, an oversight in the facilities required, and stakeholders who were not involved and whose needs therefore never entered the document.
Every one of those is a scope definition failure, discoverable at inception, by reading. None requires a site visit or a specialist.
The wider evidence, and how far it goes
The Green Book guidance is British, twenty years old, and drawn from UK public procurement. It would be reasonable to ask whether it travels.
The largest body of comparable work is Bent Flyvbjerg's, and it is blunter. His stated iron law of megaprojects is that they run over budget, over time, under benefits, over and over. He reports that nine out of ten have cost overruns, that overruns of up to 50 per cent in real terms are common and above 50 per cent not uncommon, and that the pattern holds across the countries and decades for which comparable data exist.
His prescription is reference class forecasting, building the estimate from the outcomes of comparable completed projects rather than from a bottom-up build-up of the project in front of you. It is the same move the Green Book table makes, generalised.
Both are worth treating carefully rather than quoting as a verdict. Flyvbjerg's samples are weighted towards transport and public infrastructure, definitions of overrun vary between studies, and the baseline a project is measured against is itself a choice. What survives all of that is the direction and the persistence, which is what makes it useful at inception and useless as an excuse afterwards.
The gap in the guidance itself
There is one part of the document that receives almost no attention and is, from an owner's point of view, the most important thing in it.
Mott MacDonald was unable to recommend upper and lower bounds for operating expenditure, except for outsourcing, or for benefits shortfall. The reason given is a lack of available data.
Read that again in the context of what a capital programme is for. The canonical published guidance on systematic over-optimism can tell you, by project type, how much you are likely to be underestimating the cost of building the thing. It can tell you almost nothing about how much you are underestimating the cost of owning it, or how much you are overestimating what it will deliver.
The guidance handles this honestly. It says appraisers should still consider optimism in those parameters, and should use sensitivity analysis to find the switching values, the points at which the decision would change. By how much can benefits fall short before this stops being worthwhile. How far can operating costs rise.
Those are the right questions and they produce a range rather than an adjustment, which is a weaker instrument. Two decades on, the asymmetry has not closed: the construction cost of a public project is one of the better-studied quantities in economics, and its operating cost is not.
Why the operating cost has no owner at this stage
, read: You cannot buy an outcome you cannot specifyApplying this where announcements precede definition
The Gulf has sanctioned capital programmes at a pace and scale with few parallels, and frequently in a sequence that inverts the one the guidance assumes.
The Green Book method presumes an estimate produced at outline business case, then adjusted upward, then defended. Where a headline figure is public before the definition supporting it exists, that adjustment has nowhere to go. Applying a 51 per cent uplift to a number that has already been announced is not an appraisal step, it is a political event, and the appraiser knows it before they start.
What remains available, and is worth more than the arithmetic, is the contributory factor analysis. An organisation can go through the list for a programme already sanctioned and establish which factors were mitigated and which were assumed away. That produces no new number. It produces an accurate account of how much of the original estimate was evidence and how much was hope, which is the thing a board needs before it decides whether to continue, descope or stop.
The framework for this stage sets out how to examine the estimate and the definition beneath it.
For a programme not yet sanctioned, the cheapest useful discipline is the one the guidance actually prescribes and nobody performs. Start at the upper bound. Write down what you have evidence for. Reduce only that much.
Sources. HM Treasury, Supplementary Green Book Guidance, Optimism Bias, Table 1 for the adjustment ranges, Tables 2 to 4 for the contributory factor contributions, paragraph 3.3 on the outline business case stage, paragraph 3.17 on independently verified evidence, and paragraph 4.1 on the absence of recommended bounds for operating expenditure and benefits shortfall. The guidance was prepared from Mott MacDonald, Review of Large Public Procurement in the UK, 2002. Bent Flyvbjerg, What You Should Know About Megaprojects, and Why: An Overview, and Introduction, The Iron Law of Megaproject Management, for the overrun frequency and for reference class forecasting. AACE International Recommended Practice 56R-08 for estimate classification.
Related reading
Doing the project right, and doing the right project
Almost every instrument the industry owns answers the first question. Estimating, cost control, scheduling, gateways, technical assurance, internal audit, all of them examine execution. Whether this should have been the project at all is settled once, early, by fewer people, and becomes progressively harder to ask.
ReadWhether it was worth doing is the least evaluated question
The UK government reviewed evaluation across the 227 largest projects it runs, together worth £834 billion. A third have a plan capable of finding out whether they worked. Of the three kinds of evaluation, the one asking whether the benefits justified the cost is the weakest, and the reason is that it has to be designed before the project starts.
ReadA gate that has never been failed is not a control
Assurance before anything is built costs less and changes more than assurance anywhere else in the lifecycle. It is also the assurance almost nobody buys, and the reason is not that owners are careless. It is that inception produces very little of the sort of evidence an assurance function knows how to examine.
Read