Keeping Prediction Honest
by Danielle Fong
I base my action upon prediction. Every technologist should. I try to see how the world will be, and then try and see within that future what place I may come to hold.
So prediction is fundamentally at the heart of a technologist’s work. At the highest level, we must predict to find what work focus on, and what future to aim for.
You might then think that prediction, as a skill, is worthy of practice. And practice it gets. In living rooms, in pubs and classrooms and yearbooks and dial-in talkshows and newspapers and blogs and comment threads and slashdot and every polluted corner of our existence, you find evidence: prediction is practiced all the time.
There’s a problem. In most areas of the technologist’s pursuit, it’s easy to see whether you’ve done well. Code should compile. Planes should fly. Cars should go. Bridges should stay up. We have a lot of honesty in our discipline, much of it because we are blessed with tests that we find hard to fool.
A typical test for predictions, on the other hand, is whether the story sounds good at the pub. You make some exclamation. People nod and clap. Everyone forgets.
This would be fine if you’re just looking for some conversation. But if you are, like technologists fundamentally in the business of creating the future, it becomes lot more troublesome. We are left to ignore predictive incompetence until reality slaps us coldly across the face. We are flying blind.
Taking a cue from Trevor Blackwell, I’ve decided to inject some rigor into my life: when I make predictions, instead of casting them abstractly into the air, I’ll post them here: einfall.slinkset.com. (edit: embarrassingly, slinkset is down, and I do not have an archive. Archive.org to the rescue! http://web.archive.org/web/20090510010305/http://einfall.slinkset.com/) And I won’t delete my predictions — if they turn out wrong, I’ll keep them there, as permanent reminders to learn from.
Through accountability, honesty. Through honesty, improvement.
Thanks to Trevor Blackwell for the inspiration, and John and Brett from Slinkset for the List Hosting.
Notes: a friend of mine noted that most of my predictions seem ‘pessimistic’, in the sense that they take the form of ‘X will not Y.’ I would have to agree with him. But this is largely a byproduct of how these predictions were made – they’ve come from studying some field, working in it for a while, and coming to the creeping realization that one or more of the current approaches were doomed. Besides, much of the skill of experts comes from the ability to ignore false trails.
Further Reading: An excellent site for major predictions (often with significant wagers) is Long Bets.
Insights by Danielle Fong 
You could also use something like Inkling to ask the crowd to help predict the things on your list. There’s a lot of great ones here on your list that would make excellent prediction markets.
Thanks Nate,
I’ll give that a try. :-)
Here’s a question to ponder: How many of these predictions do you expect to be around to see decided? I expect to be around for a few, and hope (with a certain amount of rational justification) to be around for most of them (although I’ll be over 150 in 2100).
I expect to be around for most of them, actually, but there will still be a lot to learn from well before the deadline. Keeping a deadline is mostly to hook progress on to some implicit timeframe.
Writing down predictions and re-visiting them when those events take place is good so that one does not suffer from cognitive dissonance. Once you have the feedback system in place, accountability, honesty and improvement follow.
The history of prediction is influenced by those that write history.
I’ve lost count of how many things the hippies were right about, and how many of the lies of those that write history (including felons and war criminals populating the current regime) continue to propagate as common wisdom. In politics, economics, and more. Even tech (just ask Stallman).
“Through accountability, honesty. Through honesty, improvement.”
I predict a balanced review of history might improve that assumption.
starving people,
Do you have specific examples for which improvement failed to follow both honesty and accountability.
Here’s a huge example, that has world wide impact. The economic policies favored by the ‘Chicago school’, as promoted by the likes of Hayek and Friedman. Plenty of failed predictions there, lots of dishonesty and lack of accountability. See Naomi Klein’s Shock Doctrine for a healthy dose of counterpoint, then review the corporate media spin that continues unabated to this very day.
It’s a problem as old as Scipio v Cato, if not much, much older.
Here’s another, for science. Global Climate Change. Documented for decades, of intensely global significance, yet ridiculed and derided for as long, careers tarnished or worse, to this very day. Attacked by a select few vested interests, quite possibly at the expense of the planet and all that walk upon it. Any serious reader of Kuhn should note that waiting for the old guard to die off places all at great peril.
Anyone conditioned to quickly dismiss the ‘kooky’, to suspect the informed, to ignore obvious facts, they too should take note, for they are not alone in lacking a new planet of escape, should things turn out as predicted.
starving people,
I mention the efficient markets hypothesis and climate change in my predictions…
You asked for examples of failures, I provided a few. Feel free to hand wave and ignore the deep implications at your own risk.
Thanks for sharing your predictions. I never thought about the accountability and honesty of predictions. It seems to bring what is kitchen table gossip into the realms academia and science… and imbues the predictions with more power.
An interesting twenty-year old climate change prediction by Dr Hansen.