Mellers et al.
Psychological Strategies for Winning a Geopolitical Forecasting Tournament
Mellers, Ungar, Baron, Ramos, Gurcay, Fincher, Scott, Moore, Atanasov, Swift, Murray, Stone, Tetlock. 2014. (View Paper → )
Probability training corrected cognitive biases, encouraged forecasters to use reference classes, and provided forecasters with heuristics, such as averaging when multiple estimates were available. Teaming allowed forecasters to share information and discuss the rationales behind their beliefs. Tracking placed the highest performers (top 2% from Year 1) in elite teams that worked together. Results showed that probability training, team collaboration, and tracking improved both calibration and resolution. Forecasting is often viewed as a statistical problem, but forecasts can be improved with behavioural interventions. Training, teaming, and tracking are psychological interventions that dramatically increased the accuracy of forecasts. Statistical algorithms (reported elsewhere) improved the accuracy of the aggregation. Putting both statistics and psychology to work produced the best forecasts 2 years in a row
Here’s some of the insights gathered:
- Train and fine tune your understanding of probabilities:
- Consider reference classes when making predictions
- Average multiple estimates when available
- Avoid common biases like overconfidence and base-rate neglect
- Use statistical reasoning and decision trees
- Collaborate with others to forecast:
- Share information and discuss rationale
- Keep each other motivated through social interactions
- Exchange news articles and evidence
- Engage in constructive critique of different viewpoints
- Track good forecasters and put them together:
- Track forecast record, identify top performers
- Group them in a team together
- Get them working well together. High performers learn faster when surrounded by other high performers.
- Encourage them to make make multiple predictions per question