How susceptible are policy-makers to biases? Are policy-makers over-confident, under-confident, or are they “well-calibrated”? The Behavioural Insights Team (BIT) can help you to answer these questions.
BIT was created within government, for government, to deliver practical solutions to complex policy challenges using psychological, cognitive and social science-based behavioural insights. Behavioural insights help government better understand the impact of cognitive biases, which is our tendency to make flawed judgements by relying on previous experiences and preferences. Ultimately, BIT helps encourage a more realistic view of human behaviour to be adopted within decision-making. Policies and services designed using behavioural insights are being rolled out by governments across the world.
BIT delivered a masterclass for the Government Office for Science on 'Debiasing the policy-making process'. We left the session having learned 3 key points:
- Cognitive biases exist - Studies suggest that many professionals are prone to overconfidence (see BIT's report). For example, when BIT newsletter readers completed a general knowledge quiz, the average respondent answered 60% of the questions correctly but was 72% 'confident' that their answers were right. Questioning our own cognitive biases can help us understand “how well we know what we know” and help us to make more sound decisions. This applies to decision making in any situation, including in policy development.
- Solutions for debiasing at an individual-level have successfully worked in other organisations - Online “thinkgroups” encourage equal hearing of opinions. Anonymising an online document that all members of a discussion can access allows everyone to voice their ideas. This helps minimise the pressures on junior employees to conform to ideas posed by senior employees, and prevents opinions being diluted or disregarded. Another option is to use a “premortem technique”, in which you are tasked to picture an outcome that will occur 12 years into the future. The exercise challenges people to imagine that their project has failed (or died), and to identify potential reasons why. The aim of the “premortem” is to increase awareness of the project’s failure points, with the basic ethos being to better identify potential project risks before and not after failure.The Department of Transport (DfT) have worked with BIT to explore solutions such as ‘thinkgroups' and ‘premortems’ that could help avoid potential behavioural biases within DfT project delivery.
- Solutions for debiasing at a departmental-level could be implemented in government - BIT advised us to make ‘well-calibrated’ forecasts for policy outcomes. A ‘well calibrated’ individual is someone who has an accurate sense of their knowledge: they can locate exactly “how well they know what they know”. A ‘superforecaster’ is a very well-calibrated individual who maintains a high level of accuracy across a diverse range of topics. Superforecasters may be an asset in planning for future events: locating who they are and how they operate could significantly increase accurate forecasting on a range of project topics.
If this blog has made you worry about your own biases and calibration, there is no need to despair. Research into training forecasters shows that online tests are able to improve your own calibration. One study found that, after only one hour of training, superforecasters improved their forecasting accuracy by 6-11%.
If you want to know more, read this paper that discusses ‘what makes superforecasters so super’.
We at GOS are now honing our forecasting skills to ensure that like superforecasters, we are well calibrated when making decisions in the policymaking process. BIT’s masterclass taught us that biases can potentially harm policy making. However, there are solutions available, at individual and departmental levels, that can relieve biases.
Understanding well-calibrated predictions presents a great opportunity to successfully create debiased policies. This helps decision-makers better plan for the future, and ultimately, ensure that successful outcomes are achieved.