Elections happened and Intrade lost

So elections happened, and Intrade lost big.

There's no need defending them that they still had Obama as leading candidate. Here are some of their minor markets one day ago:

• Obama to win Florida - 32%
• Obama to win 320 or more electoral college votes - 26%
• and so on

Intrade massively overestimate Romney's chances and possible margin of victory and underestimated Obama's.

Meanwhile it seems that Nate Silver got 49.5 out of 50 correct, since he gave Florida exactly 50.0% chances.

Anyway, I'm sure Nate Silver will be getting more congratulations than Obama over the next few days, that's not what I want to write about.

Markets vs Models

My post from two days ago caused a bit of confusion, and perhaps I wasn't entirely clear (but look at the kittens!).

I'm not claiming that yesterday's elections were a definite resolution of markets vs models question. Such resolution is in principle impossible.

Even if Nate Silver gave Obama 99% chances, and Intrade gave him 1%, even that wouldn't really be definite.

First - we have a independent sample size of one. There were multiple predictions made by both Intrade and Nate Silver, but they are not independent at all. By any measure probability of winning Ohio and probability of winning elections were very highly correlated, so we have just one observation.

Second - even if both Intrade and Nate Silver keep predicting subsequent elections, these are not independent events at all! Pollsters, polling aggregators, and market participants will keep changing their methodology based on how their previous predictions worked, so every new prediction is dependent on all previous predictions.

And third and worse of all - we don't even have independent sample size of one. Market participants had full access to Nate Silver's predictions, and could have priced them in, so that's not independent at all.

It's somewhat better in the other direction since Nate Silver claims to have made his model long time ago, and not to have made any tweaks, but then Intrade numbers were widely quoted, and they might have affected both voters and polling organizations, affecting inputs to Nate Silver's model. Compared with other problems this one is probably the least important, but in principle nothing at all is independent!

What was Intrade thinking?

Intrade numbers were so strangely optimistic about Romney it's difficult to take them at face value. Any of these alternatives makes more sense than Intrade traders honestly and rationally expecting Romney to have 30% chance:

• Deliberate market manipulation on Intrade
• Completely irrational bubble
• Traders believing in some kind of vast left-wing conspiracy by pollsters, who all lie about voters' preferences 
• Traders believing in some kind of vast right-wing conspiracy by voting machine manufacturers or some other kind of massive pro-Romney voting fraud

I'm not joking about these two conspiracy theories - they were both pretty widely discussed all over Internet and apparently some "fair and balanced" cable news stations.

Nate Silver's model actually included some small voting fraud effect of Republican voter suppression efforts, but Intrade traders would have to believe in something much greater than that to reach these numbers.

I think the most likely answer to what happened was some combination of market manipulation and bubble. We know there were attempts to manipulate Intrade by Republicans in both 2012 and 2008 elections.

In theory such efforts should always fail - if there are enough people who care only about making money - and this tends to be a safe assumption on most markets - they'd take advantage of such manipulation attempts and market would go back to rational equilibrium.

However if there's a sufficiently large number of people who take high Intrade prices as solid evidence that actual chances must be high - they can create positive feedback which turns any movement (either random or deliberate) in prices into long term price change.

This sounds speculative, but I have no other explanation for why Intrade's odds of Supreme Court overturning Obamacare increased from 2:1 to 4:1 while nothing whatsoever was happening! Someone pushed prices for whatever reason, people saw that prices are increasing and assumed that this someone has either insider knowledge or some kind of insight, and jumped the bandwagon.

No kind of rational market process can generate both 2:1 odds and 4:1 odds with zero new information.

Now stupid kind of market manipulation - throwing ton of money on the market very quickly - will probably end up being defeated since it's too obvious to everyone. But if you push the market slowly then you might very well generate a lot more positive than negative feedback.

The case for manipulating prediction markets

That's for some speculation on why market manipulation might work in practice better than in theory, but in leaves a big question - why?

After all even if you successfully manipulated the market, you're very likely to still end up losing money on election day, and most voters don't pay much attention to Intrade.

But there's one route how Intrade odds can plausibly affect the results:

• Candidate manipulates his odds on Intrade
• Candidate appears more likely to win to donors, volunteers, journalists and so on
• Candidate gets a lot more donations and coverage
• More campaign money and more coverage affect voters
• Polling shows this
• And there's positive feedback look

I don't think this would matter too much in presidential elections, but in primaries perceptions of candidates' seriousness is a huge deal. Intrade's preferred candidate Mitt Romney won against everybody else even though he never lead in any polls, and it was clear that he was very far from what the Republican base wanted.

So was Romney campaign or his supporters manipulating Intrade during primaries (and later during elections)? I doubt we'll ever see any hard evidence, but this scenario is broadly consistent with data.

I'll just leave it here as an interesting scenario to think about, not any serious conspiracy theory.

What does it mean for the future?

Actually not that much.

After this loss, Intrade traders in future elections will probably pay a lot more attention to models like Nate Silver's and election fundamentals, and will be much less likely to jump on any bandwagon of either deliberate market manipulation or random bubble.

Unless you're willing to outspend every single market participant put together, successful manipulation is only possible if you can get bubble going, and traders will probably now be much less willing to put too much weight on Intrade prices.

For prediction markets enthusiasts, this isn't even much of a problem in theory - Intrade is very far from a perfect market, with all kinds of transaction costs, and other limitations, so even if Intrade was definitely proven to be biased and bubble-prone (and we have no such proof, just some suggestive data points) - they can keep insisting better prediction market would still be "rational".

Like belief in Efficient Market Hypothesis, belief in rationality of prediction markets is simply immune to data.

Anyway, that's probably the final post about politics, I'll be back to your regular kittens and technology posting soon. (especially once Raspberry Pi I ordered arrives)

Tomorrow is the most important election in history of the future

Tomorrow millions of Americans will choose between rule guided by wisdom of the crowd embodied in free markets and rule guided by wisdom of elite experts embodied in technocrats' statistical models.

And by that I mean obviously Intrade vs Nate Silver.

What government is all about

Make no mistake - this isn't some obscure issue a few wonky bloggers care about - this is one of the biggest issues the future will face - the problem of predicting itself. Governing is fundamentally about just two things:

• Determining how you value various possible outcomes - how much do you care about low unemployment vs economic growth vs showing off your military power vs not offending God with too much gayness vs other such important outcomes of government policy.
• Predicting how various choices made by the government will affect these outcomes.

Whenever there's full agreement on both, there's really no controversy as for what government should do. You apply Bayes' Theorem and it will tell you precisely which of many possible policies to adopt.

Whenever there is disagreement, it must mean either different values, or different predictions of the future, or both.

Values

Contrary to what you'd expect agreement on values is the easy part. Most things affected by government policy are economic in nature, and people mostly agree on the same things being good - like higher incomes, lower unemployment, cheaper energy, better education, healthier environment, less poverty, and so on - they might disagree on relative weights to be given to these issues, but there's a natural utility metric - one happy voter is one point of utility - so you can just estimate how median voter feels about various issues to get a pretty good starting point for the compromise.

This won't resolve some conflicts, like the one between "people against murdering unborn babies" vs "people against forcing women to carry their rapists' babies", or whatever were the politically correct terms for them these days. Well, too bad, but there's literally no solution that will make everybody happy anyway. Such problems are fortunately fairly rare, and it's not like the government is doing a great job handling them anyway.

Predictions

A much more common and more important source of disagreements about the future is with predicting outcomes of various policies.

If one seriously believes that tax cuts for the rich lead to economic renaissance, and that welfare just makes people lazy, then Romney's policy makes perfect sense.

And if one believes that Obamacare will make healthcare available to everyone at lower cost, and that massive budget deficits saved the world from the Second Great Depression, then Obama's policy makes perfect sense.

There's little difference in values here - both Romney and Obama and most of their supporters want high economic growth, high employment, affordable healthcare, and fewer financial crises - but they have very different ideas on what's the best way to reach such goals.

How to predict the future?

So the problem of governing reduces to problem of predicting the future. And only two serious approaches exist - prediction markets and experts' statistical models.

Prediction markets like Intrade, or for that matter - stock and bond markets - sound a lot like gambling to many, but that couldn't be further from truth. Every transaction on prediction markets is a disagreement about the future - with winner taking loser's money. If you truly believed that Ron Paul was going to win Republican primaries, or that Nokia's N series was going to destroy iPhones, or that deficit spending would result in hyperinflation, you can put your money where your mouth is (on  respectively Intrade, stock market, and bond market) and if you were right then you could be an overnight millionaire. Or if you were wrong, you'd lose your money.

This is a pretty good motivation, and prediction markets tend to be pretty good most of the time. It's really most unfortunate that we don't have any widely traded prediction markets for economic growth, unemployment, and other such macroeconomic indicators, but markets we have tend to perform quite well.

The only second approach is statistical modeling based on reference class forecasting. You accumulate a lot of data about the past, and based on assumption that the future is likely to be broadly similar to the past you can make some really good predictions. This is far from automatic exercise and requires a lot of judgment as to which variables are relevant and how they might be connected, and in the end you often get pretty wide margin of error - but if reference class forecasting tells you we don't know better that's probably because we don't.

I'm completely ignoring all so called "experts" who don't base their predictions on any statistical models and just talk out of their ass, and all claims of "markets" that are not backed by actual money on the claim. These have track record more dismal than asking my cat's opinion.

So how do we decide between prediction markets and experts' statistical models?

Intrade vs Nate Silver

Unfortunately we have very few serious statistical models, and very few serious prediction markets, and even less overlap. Pretty much the only area where we can compare them against each other is predicting outcomes of American elections. And unfortunately for us, in the past they've been mostly in very good agreement with each other.

However, something really interesting happened. As of today, prediction markets and statistical models disagree more than at any time in history. Here's quick summary of Obama's vs Romney's chances of victory (normalized to 100%):

• Intrade (popular vote by at least 0.5%) - 58.1% to 39.1% (1.48 to 1)
• Intrade (electoral college) - 67.3% to 32.7% (2.06 to 1)
• Intrade (Ohio) - 68.7% to 31.3% (2.20 to 1) 
• Iowa Electronic Markets (popular vote) - 74.0% to 26.0% (2.85 to 1)
• betfair (electoral college) - 79.3% to 20.7% (3.83 to 1)
• 538 (popular vote) -  80.2% to 19.8% (4.05 to 1)
• 538 (electoral college) - 86.3% to 13.7% (6.30 to 1)
• 538 (Ohio) - 87% to 13% (6.70 to 1)

Don't be misled by the fact that they both predict Obama's lead - they differ so drastically in predicting how big is the lead it's as if they were seeing different elections. 538 offers odds more than three times Intrade's. Difference is very similar for popular vote and for Ohio winner.

If Romney wins after all, this will be a stunning victory of prediction markets over experts' models, and Nate Silver will most likely go back to baseball statistics.

If Obama wins, especially if he wins by a big margin, it will be a major defeat of prediction markets, especially Intrade.

Now you may not think a single such outcome matters much, but I have a bunch of completely unscientific anecdotes regarding their past performance.

Intrade and Obamacare Supreme Court Judgment

Intrade was really bad at predicting Supreme Court judgment, it peaked at about 80%. And Intrade turned out to be completely wrong.

The worst thing about is was not so much wrong prediction - it was how it went up from 65% (2:1) to 80% (4:1), doubling the odds, with zero new information becoming available. It was just Intrade bubble - people were estimating odds higher because they were going higher, unless someone had insider information to Supreme Court justices, which in this case at least we can safely say didn't happen.

It completely baffled Nate Silver as well, who responded with much more reasonable 1:1 odds based on statistical analysis of past information, and turned out to be much closer to being right.

Intrade and Gadaffi

Here's even more baffling one - market that Muamar Gaddafi will no longer be leader of Libya by 31 December 2011.

Not only it didn't go up to 100% after fall of Tripoli, when Gaddafi was reduced to being a mayor of Sirte at most, it actually fell down, because it turned out into market for "what the hell Intrade admins meant by leader of Libya", and it stayed there for days.

It wouldn't be the weirdest thing to ever happen if some rebels with big Intrade investment made sure the market was judged correctly by putting a few bullets in Gaddafi's head.

This shows how difficult it is to run prediction markets for anything will less defined outcomes than winners of Oscars or elections. Uncertainty over interpretation of market rules might be greater than uncertainty over interpretation of outcomes. And unfortunately most important things in life are less clear than election winners, so if prediction markets can't handle them, they're of not that much use.

Republican Primaries

Here's a very interesting case. Intrade was always far more pro-Romney than polls during primaries.

Polls disagreed widely on who was leading Republican primaries - with just about everyone including Herman Cain having their time as the top candidate - except Romney who was always number two. Meanwhile Intrade didn't doubt Romney for a moment, and Intrade was right.

Nate Silver even wrote a post why on we shouldn't underestimate Herman Cain's chances, which looks really silly in retrospect.

Tomorrow

Regardless of tomorrow's winner - be it Nate Silver/Obama or Intrade/Romney - we will learn something important about which way of predicting the future is more reliable - and I'm sure the future will care far more about this than about pretty minimal differences between Obama and Romney.

Post Scriptum

To make things even more clear, as of Tuesday morning UK time, 538 updated its predictions to 92.0% to 8.0% (11.5 : 1).

Intrade and Betfair are completely unmoved. (I don't actually know how often Betfair updates its odds, but Intrade is live).