What is reference class forecasting?
Definition and explanation
Reference class forecasting is a method of predicting the future by looking at similar past situations and their outcomes. Kahneman and Tversky found that human judgment is generally optimistic due to overconfidence and biases. People tend to underestimate the costs, completion times, and risks of their actions, whereas they tend to overestimate the benefits. Such errors are caused by actors taking an "inside view", assessing the information readily available to them, and neglecting other considerations. Taking the “outside view” — looking at events similar to ours, that have occurred in the past — helps us reduce bias and make more accurate predictions.
This brings us to the fun game called reference class tennis. Often discussions involving reference class forecasting spiral into discussions of which reference class a given situation should be in (“it’s in the reference class of books, which usually take a year to write”, “no, that’s too general, it’s in the class of cookbooks, which on average take 5+ years”).
Examples of reference class forecasting
Kahneman and Tversky were editing a textbook on psychology when they first encountered this phenomenon. In predicting how long it would take, they initially used the inside view to add up what would need to be done (collect submissions from authors, submit to publishers etc.), and predicted it would be done in a few months. It took over 7 years from that prediction, something they would have been able to work out if they had used the outside view and looked at how long other textbooks took.
Another example of reference class forecasting is predicting how long it will take you to pack for a holiday. The inside view is adding up the individual components to produce an estimate (half a day). But this estimate is neglecting the things that aren’t in your model, like that you’ll need to drive to the gas station, or that you forgot to get the marshmallows (and it’s NOT a holiday without marshmallows). A more accurate way to combat our tendency to underestimate how long it will take (the planning fallacy), is to look at how long it took you to pack for similar holidays you or others have been on. Note that Parkinson's law could also be at work, the adage that "work expands so as to fill the time available for its completion."
I used inside view when trying to calculate how long it would take me to write all these concepts, and predicted a day or two. Turns out that was way off! Much better would have been to write a few, time how long it took, and then extrapolate that out to the rest.
It’s also not just time, but any variable that you’re trying to predict or estimate: cost, number of medals you’ll win at the Olympics, likelihood of a political uprising in Tunisia. You’ve been involved in a project (at work, school or privately) that is well over budget. When setting the initial budget, if people are using the inside view, they’re likely to underestimate how much it will cost. Use the outside view to make more accurate predictions.
Also check out
Superforecasting, Philip Tetlock and Dan Gardner
Base rate neglect, Wikipedia