The benefit cost ratio – a victim of its own success
9 June 2022
The benefit cost ratio (or BCR) of transport projects continues to receive a poor press. The 2021 Institute for Government report ‘How governments use evidence to make transport policy’ said: “While the BCRs calculated are comprehensive, with this sort of appraisal, the focus on this single number can hide more than it illuminates”.
In that respect, the BCR has probably been a victim of its own success. Whereas twenty-five years ago the quantified (dis)benefits of a project were limited to analyses of travel times, vehicle operating costs and accidents, this quite short list of quantified and monetized impacts in the BCR has grown since to reflect, for instance, wider economic benefits, landscape impacts, noise, air quality and health benefits. A reasonable question is whether some of these impacts should really be monetized, if the methods and input parameters are sufficiently understood by practitioners, and if this growth in monetizing impacts has reduced the perceived necessity to provide a qualified strategic narrative rather than a single, precise number.
Some suspect that, given the complexity of transport modelling processes, the single value represented by the BCR is easily gamed by a careful choice of assumptions, many hidden in the depths of the model. I would say that this concern can be quite easily addressed by insisting on forensic reporting of model structure, parameters and other assumptions, and carried out through peer review and carefully designed sensitivity testing.
What I do consider a fair criticism of BCRs is that we end up measuring and valuing what we can measure and value, not what we should. For instance, are travel time (dis)benefits a good enough and defensible proxy for all other impacts resulting from how each individual chooses to use or trade these benefits for other medium to long term choices, such as where to live?
Which is a fair segue to another weakness, that subjectively chosen boundaries in modelling and appraisal may miss important responses to the project or policy that should also have been incorporated. Examples are unintended consequences outside of the assumed geographical model boundaries, or outside of transport’s immediate domain. But it could also be that the impacts of plausible social or economic policies on the transport system in the future are not recognized and captured.
And finally, as estimated benefits are esoteric, transitory or plainly distrusted, predicted costs become the driver of the BCR (and as real-life occasions have shown, these can be gamed and have been challenged).
UK guidance in TAG is clear, in that only options with a strong strategic case should be short listed for detailed cost benefit analysis, and the BCR will only be calculated for options which pass this test. The ATAP guidelines copy that: “a top-down strategic focus – with bottom-up information”. The BCR is a useful metric to assess variants of options that meet the strategic needs for a project, no more and no less, and ideally to compare like-for-like projects.
There is definitely a case to make for more sensitivity testing and the modelling of alternative scenarios to support any BCR, but a range of values won’t make the life of decision-makers any easier. I like to refer to a quote ascribed to Lyndon B Johnson: “Ranges are for cattle. Give me a number”. BCRs have been that number and the pressure for a single all-encompassing figure will remain. It’s up to us as transport planning practitioners to lead that debate, which includes much better visualization of all benefits (monetized or not) and a recognition of the often false precision with which BCRs have previously been presented.
Tom van Vuren is Regional Director UK & Europe at Veitch Lister Consulting, a Visiting Professor at the Institute for Transport Studies at the University of Leeds and Policy Director at the Transport Planning Society