Divide asked in comments to my post "Which crackpot cult to join" why I didn't make any snarky remarks about Eliezer and the rest of the Overcoming Bias crew. Well, here they come.
One of the most peculiar beliefs shared by the Overcoming Bias crew and most of the readers (as estimated by the comments) but hardly anyone else is "Aumann's agreement theorem", which informally says:
If two people are perfect Bayesian rationalists, they cannot agree to disagree.Like most Bayesian theorems it's mathematically flawless, the problems start when you try to apply it to the actual world. The bastardized version which is really popular on Overcoming Bias is:
After discussion between two non-perfect Bayesian rationalists, their views should be expected to be closer to convergence than before.It sounds quite reasonable - the more pro-X person shared some pro-X evidence, the more anti-X person shared some anti-X evidence, and how could knowing some extra pro-X evidence could possibly make you more anti-X than before or vice versa? At worst it will make no difference, unless someone was really horrible at discussing.
Still with me? Great! Now let's apply that to Robin Hanson and effectiveness of healthcare. As you might already know if you're a regular OB reader, Robin Hanson believes that healthcare spending is largely wasted and ineffective, he also blogs about it a lot. So according to bastardized Aumann's theorem after reading some of his posts I should be less convinced about healthcare than before, right? Yet it had the opposite effect on me, and I believe my reaction is rationally Bayesian.
Discussion is not random evidence seekingThe problem with bastardized Aumann is that it models discussion as evidence discovery. If I did random research and found out result more consistent with healthcare not working, this would indeed make me less convinced about efficiency healthcare. But discussion is not unbiased evidence discovery! Here's a model I like a lot better:
- P(healthcare works) = 50% - or any other number, it only matters in which direction it goes
- P(Robin skillful at discussing) = 90% - I have little reason to suspect lack of skill on Robin's part
- P(good evidence against|NOT healthcare works) = 90% - if it doesn't work, there should be some good evidence against it
- P(good evidence against|healthcare works) = 10% - if it works, there will probably be no good evidence against it working, weak evidence will almost certainly exist
- P(Robin's posts convincing|Robin skillful at discussing AND good evidence against) = 90% - if good evidence exists, and Robin is skillful enough, he will most likely use it correctly and his blog posts will most likely be convincing
- P(Robin's posts convincing|NOT Robin skillful at discussing OR NOT good evidence against) = 10% - if good evidence doesn't exist, or alternatively Robin fails at arguing, his blog posts will most likely not be convincing
- P(Robin skillful at discussing|NOT Robin's posts convincing) = 83.3% - down from 90%
- P(good evidence against|NOT Robin's posts convincing) = 16.7% - very significantly down from 50%
- P(healthcare works|NOT Robin's posts convincing) = 76.7% - significantly up from 50%
- P(Robin skillful at discussing|NOT Bad Robin's posts convincing) = 5.8% - down from 10%
- P(good evidence against|NOT Bad Robin's posts convincing) = 47.7% - slightly down from 50%
- P(healthcare works|NOT Bad Robin's posts convincing) = 51.9% - very slightly up from 50%
If someone who's good at convincing tries to convince you about X and fails, it is a good evidence against X.This effect is only strong when you have reasons to believe convincing arguments would be used. If you expect appeal to emotion or other low value arguments (like when a politician speaks to uneducated voters), and get it, it's really low evidence (but still against). If you expect to get strongest arguments available, but get something very weak, that's some very good evidence against.