Skip to content
Projects Advisors

02 Delivery & Controls9 min read

The score that is not a number

A technical score on a qualitative criterion records what evaluators judged. The weighting decides how much of the award that judgement controls; it does not turn the judgement into a measurement. What makes the score answerable is a record that a reader outside the room can test against the solicitation and the proposal, and the instruments examined here require different parts of that record.

14%

of US federal bid protests decided on the merits in fiscal year 2025 were sustained, 53 of 380; GAO names unreasonable technical evaluation among three prevalent grounds for sustaining protests

Stage
02 · Delivery & Controls
Published
Reading
9 min read

A tender gives 20 points to the proposed team. One bidder scores 14, another 17. The three points may be the right answer. What matters for everything that happens after the award is a narrower question: can anybody who was not on the panel find out what the 17 was given for, and what the 14 lacked?

What a technical score records

Some scores are produced by a defined calculation. A price scored by a published formula, or a criterion that awards points for a stated count or a stated threshold, can be recomputed by anyone holding the inputs, and two readers who do the arithmetic get the same answer. Other scores record a judgement: the quality of a methodology, the understanding shown of the scope, the strength of a team. There the number is the evaluators' conclusion written in digits. It is not less legitimate for that, and no instrument read here forbids it. It is a different kind of record, and it needs different support if anyone is to examine it.

The Saudi law's own wording allows for both. Non-price criteria must be objective and, to the extent practicable, quantifiable, which leaves room for criteria that cannot be quantified.

What the instruments require

Read side by side, the three do different work. The directive bounds the criterion and ties it to information that can be verified. The US regulation requires the reasons behind the evaluation to be written into the file. The Saudi law requires criteria stated in advance, a minute that gives the reasons for each opinion the committee reached, each losing bidder's own technical scores, and a binding forum. In the provisions quoted here it does not require a written reason for each score: the losing bidder is told the reasons for its exclusion and its technical scores, not the reasons for each score.

Why the weighting does not remove the judgement

A weighting converts a score into a share of the award. Twenty points out of 70 on the team decides how far the panel's view of the team can move the result. It says nothing about how the panel formed that view. The economic meaning of a weighting, what one point costs in price, is worked through in its own piece.

What a technical point costs under a published weighting

, read: Three award rules make you disclose how the criteria are weighed. None makes you defend it.

What matters here is the direction of travel. The weighting multiplies whatever the score contains. A score resting on stated evidence becomes a large, defensible part of the award. A score resting on an impression becomes a large part of the award too, and the arithmetic around it makes it look as firm as the price.

What a reviewable record contains

Five things, and they are distinct. A system can have one without the next.

  1. The criterion and its weighting are published. The bidder knows what will be scored and how much it counts.
  2. The evidence that earns a score is defined. The solicitation says what a bidder must show, submit or commit to for the higher mark: named people with stated experience, a method that addresses named risks, a programme with stated durations.
  3. The reasons for the score are recorded. The file says what the evaluators found in the proposal against that evidence: the strength, the weakness, the missing commitment.
  4. The affected party and the review body can see the reasons. A score that is disclosed without its reasons can be disputed only in the abstract.
  5. The reasons can be tested against the solicitation and the proposal. A reader outside the panel can put the recorded reason beside the page of the proposal it refers to and the criterion it applies, and say whether they agree.

The 14 and the 17 become reviewable at step 3 and contestable at step 4. Before that, the losing bidder and the review body have a number and nothing to test it against.

What protest review can establish

GAO's annual report to Congress gives the figures for the US federal forum. In fiscal year 2025 GAO received 1,688 cases, 1,617 of them protests. It issued 380 decisions on the merits and sustained 53, a sustain rate of 14%. Its effectiveness rate, the share of protests closed in which the protester obtained some relief, either because GAO sustained the protest or because the agency took voluntary corrective action, was 52%. GAO names three "most prevalent reasons for sustaining protests": unreasonable technical evaluation, unreasonable cost or price evaluation and unreasonable rejection of a proposal. It numbers them in its list without saying that the order is a ranking, and gives no count for each. Its example of the technical ground: the agency credited the awardee with staffing for the required 11-month period when the awardee had proposed staffing for 9 months.

The example is worth reading for what made it decidable. The record said what the evaluators credited, the proposal said something else, and the solicitation set the requirement. Each was a document, and the error was the distance between them. That is step 5 in operation: a reason recorded clearly enough to be wrong in a way somebody could show.

The two rates measure different things and should not be run together. The 14% is a share of merit decisions. The 52% is a share of protests closed and includes cases where the agency changed course before any decision. GAO also counts docket numbers rather than procurements, so a supplemental protest on the same award is a separate case.

What review cannot establish

Review tests a recorded reason against documents. It does not supply a second, independent measure of quality against which the panel's judgement can be checked. Where the reason says the methodology was stronger because it addressed a named risk and the proposal does address it, a reviewer can confirm consistency and little more. A bounded judgement with a sound record is still a judgement.

Nor do the figures describe evaluations nobody protested. A sustain rate is a rate among cases brought and decided, and GAO states that agencies need not report their reasons for taking corrective action. A low rate of overturned technical evaluations can mean sound evaluations, or records too thin to contest, and the statistics alone cannot say which.

For the Saudi system the same caution runs further. The law provides the forum and the Regulations provide for its decisions to be made available. How the committee treats technical scores, and how completely its decisions can be read in practice, are not answered by the text of either.

What to examine

  • Take one judgement-based criterion and ask what a bidder would have had to submit to move from 14 to 17. If the solicitation cannot answer, the criterion measures preference, and the weighting multiplies it.
  • For a criterion on the team, apply the directive's test: can the quality of the staff assigned have a significant impact on the performance of this contract?
  • Ask for the evaluation minute. Check whether it gives reasons for scores or only for the committee's recommendations and any dissent, which is what Article 45(3) requires.
  • Take one recorded reason and find the page of the proposal it refers to. If the reason cannot be located in the proposal, the record cannot be reviewed, whatever the score.
  • Check what a losing bidder received under Regulations Article 85(2): its technical scores, and whether anything explained them.
  • Look for the Article 86 committee's past decisions where the Regulations say they are made available, and whether any of them turned on a technical evaluation.

Sources. Saudi Government Tenders and Procurement Law, Royal Decree M/128 of 2019, Articles 24, 25, 45, 53, 86 and 87, and its Executive Regulations, Articles 8 and 85, in the consolidated Arabic text published by the Ministry of Finance, with the Ministry's English translation used as a reading aid. Both remain in force until the law gazetted on 4 September 2026 takes effect, 120 days after publication. Directive 2014/24/EU, Article 67, Official Journal text at EUR-Lex. Federal Acquisition Regulation 15.305, FAC 2026-01, at Acquisition.gov. GAO, Bid Protest Annual Report to Congress for Fiscal Year 2025, B-158766, 12 December 2025.

Cite this page

Projects Advisors, “The score that is not a number,” published 2026-09-25, https://projects-advisors.com/insights/the-score-that-is-not-a-number

ShareLinkedInX