Superforecasting

• Superforecasting: The Art and Science of Prediction
• Author: Philip Tetlock with Dan Gardner
• Publisher: RH Books
• Publication date: 2015

What is the future of your business? What are the emerging trends that will shape your market? What will your category look like in 5 years? In the universally acclaimed Superforecasting: The Art and Science of Prediction, social psychologist Philip Tetlock has some interesting answers in this manual for seeing the future in an uncertain world.

First, you are probably asking the wrong question. Long term predictions – typically anything more than a year out – are notoriously inaccurate. The world is like weather, short term predictions can be accurate but long range forecasts are virtually useless because of uncertainties and unknowns.  So the first message of Superforecasting is sobering; the predictions of futurologists, trendologists, experts and opinion leaders are barely better than a flip of a coin, or in Tetlock’s words as “as accurate as a dart-throwing chimpanzee”. If you must employ an agency or an expert to predict the future, at least ask them for their track record first.

Second, how predictable something is depends not only on how far into the future that thing is, the circumstances and the nature of the thing itself, but also who is making the prediction.  Some people are systematically better at making predictions than others. And the usual suspects – the opinion leaders, experts and professional forecasters – are not your best bet.  The art and science of seeing the future is grounded in thinking style, not expertise. Foresight requires intelligence, numeracy (predicting involves thinking in terms of probabilities) and determination, but above all open-minded critical-thinking. These traits were identified by Tetlock in his analysis of the top 2% of predictors in an open prediction tournament called the Good Judgement Project. Organised by Tetlock and financed by the US military, this study confirmed his earlier study of 82,361 professional predictions that concluded that political experts and analysts were no better than predicting political outcomes than the market research industry. The problem with opinion leaders is they have opinions, and these cloud judgements and promote closed mindedness.

A superforecaster’s thinking style, on the other hand, is based on open-mindedness; open-mindedness is three times as powerful a predictor of forecasting ability than its closest rivals – intelligence and numeracy. One key aspect of this open-mindedness is openness to update and change one’s own beliefs, judgements and predictions. Forecasters learn to become superforecasters by learning from their forecasting experience. They live in a state of perpetual beta. No. Fixed. Beliefs. Another key aspect of open-mindedness is to think in terms of probabilities rather than definites or absolutes; certainty is kryptonite for forecasting. However open-mindedness only works when it goes hand in hand with critical thinking, which means being critical of the evidence, biases and assumptions, including one’s own. Critically, critical thinking means being self-critical. It also means using logic, reason and evidence instead of beliefs, assumptions and gut-feel. Put open-mindedness together with evidence-based critical thinking, and throw in some intelligence, numeracy and probabilistic reasoning – and you have the ingredients to become a super forecaster

Superforceasting concludes with 10 evidence-based recommendations to improve your own superforecasting skills

1. Triage: Sort the predictable from the unpredictable, and don’t waste time trying to predict the unpredictable. That means consigning impenetrable ’cloud-like’ questions that are long term and vague to the bin. If you’re going to predict the future keep it short-term and specific
2. Break seemingly intractable problems into tractable sub-problems: Chunking one big prediction problem into a series of sub-problems can improve prediction accuracy and flush out unknowns
3. Strike the right balance between inside and outside views: Superforecasters don’t look at a problem only in its own terms and context, they use ‘outside’ analogical thinking to ask themselves ‘what is this situation like?’. The future you are predicting has already happened elsewhere.
4. Strike the right balance between under- and overreacting to evidence. Belief-updating is to good forecasting as brushing and flossing are to good dental hygiene. Be prepared to shift your position when new evidence arises, but use critical thinking to question the evidence
5. Look for the clashing causal forces at work in each problem. Always look at the counterargument to your prediction, the truth is likely to be a synthesis of your thesis and the antithesis
6. Strive to distinguish as many degrees of doubt as the problem permits but no more. In plain English, give your prediction a probability rating (e.g. 70% chance of happening). The more granular the probability, the (usually) better
7. Strike the right balance between under- and overconfidence, between prudence and decisiveness. Superforecasters don’t rush to judgement, but neither do they sit on the fence. Instead, they are decisive in their decisions but realise that the underlying predictions informing those decisions are probabilistic
8. Look for the errors behind your mistakes but beware of rearview-mirror hindsight biases. One of the defining characteristics of superforecasters is to systematically conduct a forensic post-mortem on past predictions, and learn from them
9. Bring out the best in others and let others bring out the best in you. Inform your forecast with input from others by using precision questioning (helping them clarify their arguments), and constructive confrontation (disagreeing without being disagreeable).
10. Master the error-balancing bicycle. Mastering the art and science of forecasting means balancing opposing biases and errors. And like riding a bike, you can’t learn to forecast without trying. Practice will never make perfect, but it will you a better forecaster

The BG Take

We absolutely loved Superforecasting; unlike so many other business bestsellers Tetlock has more than one big idea to sell. Instead, Superforecasting is nuanced but powerful, suggesting that a learnable thinking style – involving multiple features – drives prediction skill. The challenge here for us as an agency is that we need to balance the need for nuance, caution, and modesty, with the need to create  a certain vision in an uncertain world that can act like a guiding Northern Star for innovation. That said, we’re totally sold on the learnable skills outlined by Tetlock to become better at seeing the future’. We particularly like the insight that the future you are predicting has already happened elsewhere, and the superforecaster trait or analogical thinking. Best book we’ve read in 2016 yet.