So what about prediction? What is the difference between a backward-looking approaches to decision-making and an anticipatory forward-looking ones (discussed here)? It’s partly about the difference between probability (and risk) and possibility (and deep uncertainty) and partly about people’s assumptions about what causes the future to come about. And it’s not just about the ability to model either – there are limits to that.
And, anyway, why is prediction such a challenge? I think it was Dan Quayle who is supposed to have said “The trouble with the future is that you don’t know what is going to happen!”. He’s so right …
It’s easy to assume that if we know the past, we can predict the future. But the natures of the past and of the future are not the same. The past is like a puzzle, things happen and provide facts which are like pieces of a jigsaw puzzle. If some are missing, we probably know they are the red ones with wiggly edges and we can go and look for them. What we have candidate pieces we can apply a matching algorithm against the known need. When the right piece comes up we have the solution. Certainty and predictability seem self-evident, but are they?
The diagram below illustrates this. If we have evidence (green circle) it implies an event and the presence of other evidence waiting to be discovered. Our assumptions can be based on what we know or might know. But the limits on prediction are set by the ‘Prediction Horizon’. With puzzling, we can only predict a short way ahead about, for example, people’s intentions (shown by the top of the curve, probably only seconds); but we can predict quite a long way ahead about geographical matters (the lower part of the curve). Why is that?Well. when we try to look into the future we are actually considering, not puzzles, but mysteries. A mystery is ill-formed, it has no pieces, there is no right answer or final outcome and we cannot be sure we have solved it – this is the essence of deep uncertainty. What we can do is form hypotheses about possible futures and seek indicators that support or refute the various possibilities. By ‘competing’ the hypotheses we can anticipate futures and put in place adaptive contingencies. Succesful outcomes are, therefore, practical and feasible. Look at the next chart.
Some of the possible futures take us beyond the prediction horizon because we are not relying on past evidence – instead we are using our imaginations to ‘think the unthinkable’ – to think not what is probable, but what is possible. The trick, as the third chart shows, is the way that you analyse possibilities and compete the hypotheses. This requires leadership, professionalism, intellect, imagination and courage to buck the trends – and people are doing it effectively and succeeding where others struggle.
References: The discussion of puzzles and mysteries has been adapted from Treverton, Gregory F. “Reshaping National Intelligence for an Age of Information“. Cambridge University Press (2003).