The Curious Case of Senate Filibuster

American Constitution states as clearly as it ever states anything, that to pass a law majorities of House and Senate are required, and doesn't know anything about filibuster process - this silly Senate regulation in which 60 votes are required to even bring anything to final voting.

And it would be so easy to get rid of such rule - mere 50+VP majority in Senate can change Senate procedures. Even more curiously - it would clearly be in interest of the majority party - be them Republicans or Democrats - to get rid of such silly rule. The argument that they might want to keep it for the future, as they might end up in minority later is obviously wrong, as your Democratic/Republican Senate keeping filibuster doesn't do anything to stop the next Republican/Democratic Senate from abolishing it.

Persistence of such major non-Constitutional feature which clearly favors the minority party is extremely curious.

Voting Power

The mystery will be resolved soon. For now let's step aside for a moment and imagine a Parliament of a fictional European country with four parties - Armadillo Party, Bear Party, Cat Party, and Dog Party, with 45%, 45%, 6%, and 4% representation in their Parliament. As it's a fictional country, let's assume perfect party discipline, and simple majority required to pass anything. How important are the parties?

Naively looking at their statistics, it would seem that Armadillo and Bear parties should be vastly more important than Cat Party, which should be just slightly more important than the Dog Party. This is entirely wrong. To pass anything you need one of the following sets on board:

• Armadillo Party and Bear Party
• Armadillo Party and Cat Party
• Bear Party and Cat Party
  

The Cat Party with its 6% of vote has as much power as Armadillo and Bear parties with 45% each! And Dog Party with its 4% has no power whatsoever - its support makes no difference for whether a bill will pass or not!

This highly counterintuitive result is mathematized as two "power indexes" - Banzhaf power index and Shapley-Shubik power index. They differ in minor details, but basically they tell you how important are different players in a complex voting system.

Now there's important caveat with voting power indexes - they assume interests of parties are entirely independent - Armadillo Party is as likely to agree with Bear Party as with Cat Party as with Dog Party - something which is not true in the real world at all. No see why it is important imagine a Bear-Cat coalition - having 51% of votes together they have 100% of power, while Armadillo and Dog Parties become entirely irrelevant. Why wouldn't Bear and Cat Parties want to start one? Mostly because coalition victories won't do you any good if you don't agree with other coalition members. Bears might push for taxpayer subsidies for honey production, something that Cats would much rather not have - and could shut down the idea together with Armadillos if they weren't in this silly Bear-Cat coalition. Increased coalition's voting power is usually countered by having to vote for things coalition wants that you're not terribly enthusiastic about.

Anyway, let's look at American system, both as its specified in the Constitution, and as it works with the filibuster. USA is no Europe, so we cannot assume any Party discipline - let's go further and assume every Representative's and every Senator's votes are entirely uncorrelated. This isn't true for many issues which follow left-right party lines, but is very much so for the highly essential pork barrel spending and other things they do for lobbyists in exchange for campaign contributions. The only exception to the independence rule I'm going to make is assuming that Vice-President always does what the President tells him to, and is no independent actor. If there were any votes in history in which they disagreed, I'd like to know about it.

Assuming I coded it right, Shapley-Shubik voting powers of the President, individual Senators, and individual Representatives in Constitutional system are 16.66%, 0.391%, and 0.102% respectively - president is worth almost 43 Senators, and a Senator is worth 3.8 Representatives. By the way that's why elections for President - supposedly "executive" function - focus primarily much on legislative issues. American President has large levels of control over legislative and judiciary branches of government, in addition to his executive responsibilities - as extremely unhealthy concentration of power.

Now with filibuster, the situation changes drastically - President's power drops to 9.115%, Senators' powers increase to 0.716%, and Representatives' plummet all the way to 0.044%. Now President is worth only 12.7 Senators as opposed to previous 43 - and more what's even important - each Senator is worth over 16 Representatives!

Numbers are considerably different with Banzhaf index - it's 3.996%, 0.295%, 0.153% Constitutionally and 2.468%, 0.564%, 0.094% with filibuster - but the story is the same - filibuster greatly increases Senator's powers. Knowing that it's not terribly surprising that Senators are not too eager to abolish filibuster - they would be giving away a lot of influence with it!

This result might be surprising at first, but there's straightforward intuition behind it - if there are multiple requirements for a law to pass - the one which is most difficult determines the process - Constitutionally passage in House and in Senate are equally difficult, giving each of them equal powers - but if Senate decides to make passage harder - it will be very unlikely for any bill to have 60% support in Senate without also accidentally having at least 50% in the House.

State Size

But wait - there is more! Small states are massively over-represented in Senate, while having only fair representation is House of Representatives. By unduly strengthening the Senate, filibuster increases power of small states. By how much? As situation becomes a bit too difficult for exact methods, I ran a quick Monte-Carlo simulation of Shapley-Shubik voting powers, assuming all states vote as blocks - so we have 50 states voting, each with 2 Senate votes and some number of House votes; plus independent President.

Constitutionally President has 14.88% power in such case, the most populous state California 8.08%, while the least populous states with one Representative like Wyoming has 0.79%. That's a huge unfair and undemocratic overrepresentation already - California has 69x the population of Wyoming, but merely 10x as much power. With filibuster it gets far far worse - President's power drops to 9.49% - not surprisingly since more of it goes to Senate, but now the difference between California and Wyoming is merely 5.10% to 1.34% - 3.8x. Californians are underrepresented relative to Wyomingites by a factor of 18x. Not terribly democratic, is it? Actually I'd love to discuss the point that less democratic systems like American are not in any obvious way less effective at solving real world problems than more democratic systems like let's say German - but let's leave that for another post.

Even if you're from a small state, you shouldn't be getting too happy - small state Senators are the most corrupt members of Congress, and the baseline isn't terribly good either. There's the usual disclaimer that power indexes refer to idealized situation in which everyone votes independently from everyone else - which describes pork barrel spending and miscellaneous lobbying much better than it describes issues of economy or abortion. Still, let's not be as naive as Al Gore about the Senate - if this lobbyist-infested place couldn't even get popular, life-saving, deficit-reduring, and sensible from every other possible point of view public option, what are the chances of it passing expensive and unpopular climate bill which will only benefit some Bangladeshis decades from now? Nobel Prize offers no protection from wishful thinking.

Misunderstanding statistical significance

People are natural cargo cultists - they (I'm saying "they" as I'm secretly a cat - just don't tell anyone) have natural tendency to focus too much on superficial characteristics - if something is good, everything vaguely associated with it must be good too! Or if something is bad, everything vaguely associated with it must be bad. So as people got into their heads once that Soviet Union and its "socialism" were "bad", everything "socialist" must be "bad" - like single payer healthcare, financial system regulations, and minimum wages. Never mind that even the basic premise is wrong, as performance of Communist economies was fairly decent, and as research says, Soviet growth performance was actually slightly above the global average in the period 1960-89, at 2.4 percent per year. Eastern Europe's growth was somewhat lower than USA's, Western Europe's, or Japan's, but it managed to beat by huge margins growth in capitalist countries of Latin America, South Asian, or even UK for that matter. Not to mention China vs India, or spectacularly good performance of the "socialist" Scandinavia. Or how pre-Reagan socialist USA was much better off economically than after Reaganite revolution.

Anyway, science. Two practices of science get fetishized to a nauseating degree. They are peer review and statistical significance. I cringe every time some (let's generously say "misinformed") person says that peer review is somehow central to science, and something being "peer reviewed" is somehow evidence of quality. The only thing peer review does is enforcing standards of whichever field the research is in. For example while Dialogue: A Journal of Mormon Thought peer-reviews its articles, I doubt they're worth that much more than Youtube comments. And creationists have their own peer-reviewed journals too! And in spite of widespread use of peer review, there's very little evidence that its variations affect quality in any way.

Most disciplines are not as bad as Mormon theology or young-Earth creationism, but standards of the field have nothing to do with solid science. First, there's a problem of entirely arbitrary standards which simply waste everybody's time like this and various incompatible citation rules:

I remember when I first started writing on economics, I was scolded for formatting my papers in a two-column, single-spaced format.  While that format was common in computer science, to be taken seriously in economics a paper must be formatted as single-column, double-spaced.

But all academics eventually learn those. A much greater problem than these time-wasters are standards which are actively harmful, like...

Statistical significance

Probably the single biggest problem of science is appalling quality of statistical analysis. As a funny anecdote, I remember back at the University when I had to go through a bunch of database optimization papers - which implemented various performance hacks, and measured how well they do - and more often than not they did it on completely bogus data generated just for this case. If you spend even five seconds thinking about it, it's obviously wrong - all non-trivial optimizations highly depend on characteristics of data access patterns, and differences in results are many orders of magnitude. But for one reason or another using realistic data never got to be part of "good" database optimization research.

For one historical reason or another, almost all science got affected by obsession with "statistical significance". Typical research goes as follows:

• Gather data
• Throw data at some statistical package
• Keep tweaking "hypothesis" until you get "statistically significant" answer, and send for publication
• If no "statistically significant" answers are obtained, usually forget about the entire thing, or maybe send for publications anyway and hope it gets accepted
  
• Watch people misinterpret "statistically significant" as "almost certainly true" and "not statistically significant" as "almost certainly false"
  

But of course, "statistically significant" results are false very often. Assuming perfectly run research, which is never true, 95% statistical significance only guarantees at most 1 false positive per 20 tested hypotheses (well, independent hypotheses, but let's skip complications here). So if you tweaked your hypothesis 50 times in your statistical package, your "significant" results are very likely to be wrong. Then according to established scientific practice you can publish a paper claiming that "broccoli reduces risk of cancer in Asian women aged 40-49 (p<0.05)", "global warming causes malaria (p<how dare you question Al Gore)", or somesuch.

Still, that's nothing compared with uselessness of something being "not statistically significant". Even otherwise smart people routinely misinterpret it as "with high probability not true". If so, let me start a study in which I will take wallets of randomly selected half of 10 such people, and we'll measure if there's any statistically significant reduction of wealth from me taking their wallets. Double blinded and all! And as science will prove no statistically significant reduction relative to control group, which court would dare to question science and convict me for doing so?

While false positives - wrongly rejecting null hypothesis - can come from either bad luck, or bad study; false negatives - not rejecting wrong null hypothesis - can come from either of those, or from insufficient sample size relative to effect strength. How strong would the effect need to be?

Let's get back to the smoking trial:

A randomised controlled trial of anti-smoking advice in 1445 male smokers, aged 40-59, at high risk of cardiorespiratory disease. After one year reported cigarette consumption in the intervention group (714 men) was one-quarter that of the “normal care” group (731 men); over 10 years the net reported reduction averaged 53%. The intervention group experienced less nasal obstruction, cough, dyspnoea, and loss of ventilatory function.

During the next 20 years there were 620 deaths (231 from coronary heart disease), 96 cases of lung cancer, and 159 other cancers. Comparing the intervention with the normal care group, total mortality was 7% lower, fatal coronary heart disease was 13% lower, and lung cancer (deaths+registrations) was 11% lower.

Which is not statistically significant. And some libertarian will surely use it to "prove" that smoking is perfectly fine, and it's all government conspiracy to mess in people's lives.

But what were the changes of getting statistically significant results? For clarity let's skip all statistical complications and do it the most brutal possible way, and even make both groups the same size. Control group size 720, intervention group size 720, true chance of death in control group 45%, true chance in intervention group 42% (7% reduction in mortality) - we just don't know it yet. Statistical significance levels 95%. So control group had 95% chance of getting between 298 and 343 deaths (I'll skip the issue of one-sided and two-sided tests of significance as the entire idea is highly offensive to Bayesians). Chance of intervention group having fewer deaths than 298 - merely 38%. So assuming entirely implausibly that this 1440-person-strong 20-year study was perfectly run, there's be 62% chance that the results will be worthless because 1440 people is very far from enough. Except as maybe fodder for a meta-analysis.

By the way reduction is merely 7% as what was studied was "trying to convince people to stop smoking". Most people wouldn't be convinced or would relapse; and many in control group would stop smoking on their own.

Anyway, how many people would we need for study with 45% and 42% death rates groups to have less than 5% chance of both false positive (conditional on null hypothesis being true) and false negative (conditional on null hypothesis being false)? 3350 in each group, or 7100 altogether. And that was smoking - the biggest common health risk we know. How many people would we need if we studied normal-risk people, let's say 10% death rates during study time, and levels of relative risks typical for diet/lifestyle intervention, let's say 1%? Two times 1.2M, or 2.4M people. More than the entire population of Latvia. And that's before you consider how highly non-random are dropouts from such studies, and how they will swamp out any results we would get. And any double-blindness here?

In all likelihood, we will never get any data about effects of this kind of interventions. Science will have no idea if fish oil cures cancer, vegetables lower risk of obesity, or if organic is better for you than non-organic.

Hammy the Hamster tries organic food

The only thing we will have will be lower quality indirect studies with highly ambiguous interpretations, which we might actually have to accept. That, and misunderstandings of statistical significance.

Libertarianism is externality denialism

Yesterday I wrote about libertarian blogosphere, skimming 5 posts each from what is supposedly top 20 libertarian blogs.

There seems to have been quite a few angry reactions, which I will partially address in this post.

Methodology

All ways of discussing politics are flawed, and so was the method I used. But it was hopefully flawed in a different way. It was a methodological experiment.

As my attitude towards libertarianism is not difficult to figure out, I tried to limit my bias by relying on someone else's list of top libertarian blogs, and use transparent selection procedure. At first I wanted to just take the most recent 5 posts from each blog, but there were too many link-only posts, personal stuff, meta-information, and such, which I believed would only introduce noise to the results. I didn't establish clean-cut criteria for selecting which posts qualify for inclusion, so a few decisions might have been somewhat dubious. It's inevitable when something new is tried, and I don't think it affects the results significantly. From one of the comments:

I can only assume that a certain amount of skimming was involved in garnering your links as it shouldn't come as much of a surprise that smoking would feature high. Freedom2Choose is a grass roots pro-choice organisation, set up with the express purpose of opposing bans on lifestyle choices (and before you follow the usual line - no, no funding except from working class members who are angry about the increasing restrictions and bans). The subject would therefore be naturally prominent. The Frank Davis blog was set up expressly in anger at the smoking ban (the clue is the fact that his sub-title is "Banging on about the smoking ban"), and seeing as libertarians oppose authoritarianism, and that the UK ban is as authoritarian a policy as we have seen recently, of course it's going to figure.

I assume the top 20 list is representative of libertarian blogosphere. Pro-smoking articles appeared on 4 different blogs on this list, so it's not accidental inclusion of just one or two smoking-centric blogs. And unlike with climate change which is in news right now, and therefore overrepresented, nothing new is happening that's related to smoking, so I doubt this is an artifact - smoking seems to be quite solidly one of the big libertarian subjects.

Externality denialism

One thing which I observed in both original posts, and in feedback comments I got was persistent denial of existence of externalities.

This is economics 101 diagram of any economic activity - a firm produces a product, which it gives to consumer for their money. Both production and consumption have some external costs and benefits, which they impose on others regardless of their wishes.

Libertarians are really uncomfortable with economics - at least with anything resembling modern science - they sure have their own folk version of it. They pretend that red part of the diagram doesn't exist. The thinking seems to be something like:

• I feel that governments are always and without exceptions evil
• If externalities existed, governments would actually be useful
• So I won't believe in existence of externalities
  

It starts with values, and ends with statement about reality, as if the nature cared about your values. It is of course logically invalid, but humans are extremely prone to this kind of rationalizations.

Disregarding artificial controversy influenced to large extent by industrial interests (tobacco industry was the leader of this, and this is extremely well documented), science is quite sure about existence of health risks of second hand smoking, anthropogenic climate change, and such. And equally real is plain irritation of non-smokers due to people smoking in their faces. But these facts are very uncomfortable to libertarian position, so you can find arguments like these:

You will also not find one of those posts disputing the link between smoking and cancer. Passive smoking and cancer, yes, simply for the reason that there are hugely more studies showing it to be a construct of the hysterical anti-smoking lobby shilling for the superrich pharmaceutical companies (thought you were against such things?). That debate is nowhere near over as many would have you believe.

Not one post I have read on any of the aforementioned blogs talks of non-smokers being denied the right 'not to have smoke blown in their faces'.

The fact that you use the phrase "climate change denial" shows that you believe the AGW hypothesis. Fair enough. However, if you do not believe the hypothesis (or think that it is pretty darn suspect) then cap-and-trade—for all its internalising of externalities—is a total nonsense because CO2 does not do damage to the environment (in fact, it's actively beneficial because it helps trees and plants grow). Do you see?

They not only disagree with particular solutions, such as smoking bans, or cap&trade - but deny existence of externalities in the first place.

Externalities and the Pigou Club

The mainstream economic position is that externalities are not only real but very important, and the best way to deal with them is internalizing them, usually by appropriate taxes or subsidies, or mostly equivalent ways such as cap&trade, which strangely some consider Pigovian, and some don't:

While Pigovian taxes are economically-preferred way to deal with externalities, they're not the only way and that's due to the second big concept which libertarians pretend doesn't exist - transaction costs.

Transaction costs

Now I'll get quite far away from the main subject. Smoking is harmful to others, but amount of harm depends on circumstances - smoking in your own home without anyone around is less harmful than smoking inside a train filled with people. Standard Pigovian solution would be taxing different contexts of smoking at different levels, but such scheme would be highly impractical.

Driving is also harmful to others - not only due to greenhouse gases emissions and straightforward pollution, but also because of traffic congestion it generates. Taxing driving on a rural road differently from driving in rush hours in Central London would be a good idea, and it would also be highly impractical. This is actually attempted in spite of impracticality, and very high costs of running the system with the scheme barely breaking even if capital costs are included.

So every now and then economic analysis would advise doing something more brutal than just taxation - like an outright smoking ban in public places, ban on leaded gasoline, or my favourite ban on leaded solders. Sometimes no realistic method of Pigovian taxation can be implemented, and so direct regulation is necessary.

Standard anti-Pigovian arguments

Mankiw described 4 basic reasons why someone could be against Pigovian Taxation in a logical flowchart.

1. You deny the existence of these externalities as a type of market failure. Perhaps you think you live in a Coasian fantasy world where people bargain without transaction costs to reach efficient allocations. (Note: I am not suggesting that Coase himself thought we lived in such a world—he considered it only a useful thought experiment.)

2. You recognize the externalities but you don’t think the government should try to respond to them. You are such a believer in small government that you are willing to live with inferior economic outcomes, such as pollution and congestion.

3. You recognize the externalities, think the government should try to correct them, but think the current low taxes we put on gasoline are sufficient. In this case, you have weighed and rejected the evidence, such as that of Parry and Small, that higher Pigovian would be optimal. (Parry and Small calculate an optimal tax of $1.01 for the United States in today's dollars. After my proposed phase-in of a $1 hike, the U.S. tax would be $1.40. Assuming 10 years of 3 percent inflation, the tax in real terms would approach almost exactly what Parry and Small recommend. By the way, the published version of Parry and Small was in the American Economic Review, September 2005.)

4. You recognize the externalities but think the government should try to correct the market failure through regulations (such as CAFE standards) or through market-based solutions that do not raise government revenue (such as cap-and-trade systems). Perhaps you are concerned that government would waste the extra revenue on useless government programs.

It's definitely possible to make an intelligent case against Pigovian taxes, based on #2, #3, #4, or some more creative argument - here's one by Bryan Caplan, here's another by Arnold Kling. My problem is not libertarian conclusions (that internalizing externalities is bad), but that they take the #1 route of living in fantasy land in which externalities don't exist.

Back on-topic

I strayed quite far off-topic, but off-topic asides are as central to my blogging as cats. This extended writing on externalities is due to libertarian blogosphere's obsession with externality denial - climate and smoking posts together constitute 41% of all posts.

The second most popular subject is taxation - quite related to the first one, as the point of government is largely taxing and subsidizing in a way that corrects market failures, and running parts of the economy which market is particularly bad at running - such as police, healthcare, pensions, social security and such. And at least in Britain the train system, which seems private companies seems to be completely incapable of running compared to Deutsche Bahn or even TFL.

Now it's perfectly fine position to support lower or higher levels of taxation, deficits, or spending. But what libertarians care most about?

• Lowering taxes on rich bankers, and other super-rich, even though their very jobs exist only thanks to taxpayers' bailouts. This increases deficits, which leads to...
  
• lowering deficits by layoffs and pay cuts in public sector, reducing welfare payments and such.
  

And we end up with massive transfer of wealth from everyone else to the super-rich. If that's not shilling, what is it?

By the way I'm surprised that my post was misunderstood - I wasn't trying to find out if libertarian blogosphere is any good, but mainly what it writes about. My finding that I find it boring was coincidental.

Timing of peak CO2 emissions

I previously argued (1, 2, 3) that we can expect peak CO2 emissions around 2050. Copenhagen summit draft calls for peaking before 2020. Now it's not entirely impossible, but I would say it's quite unlikely.

Here's a quick list of my assumptions, and how timing of the peak, size of warming, and likelihood of geoengineering would change if they were wrong:

• Economic development of poor countries - faster development means more emissions. As this is the main driver of energy demand for the next few decades in my analysis, much slower development of poor countries will indeed move the peak a lot earlier. The consequence of slower growth is economic genocide, and it's far worse than anything climate change could do, with or without geoengineering. It's not a theoretical possibility, economic genocide is happening right now.
• Correctness of climate science - putting Climategate fraud accusations entirely aside, IPCC has ridiculously wide confidence intervals. Their A1 scenario which is closest to what I describe here expects likely temperature growth to be between 1.4C and 6.4C by 2010, and their likely means merely a 66% confidence interval. If you know anything about science, "95% confidence" turns out to be wrong more often than not, and these estimates are merely scenarios conditional on countless assumptions, not actual forecasts. So what IPCC is saying is essentially "it will get hotter, we have no idea whatsoever by how much". If warming turns out to be on the low end of IPCC estimates, we'll likely see slower adoption of technologies to reduce CO2 emissions, and no geoengineering. If warming turns out to be on the high end of IPCC estimates, we'll likely see faster adoption of emission-reducing technologies, and massive scale geoengineering will be as inevitable as I predict. This is not as relevant as it seems as timing of CO2 peak depends on decisions we'll make in the next few decades, long before we find out 2100's warming.
  
• Technological progress - faster technological progress can result in lower emissions. If we get breakthroughs in cheap solar energy, battery technology, algae biofuels and such, it might result in much faster peak, maybe even before 2020. But I wouldn't rely on this, a lot of money was spent on these technologies and progress has been slow - what we need is not just some incremental progress which we'll undoubtedly see, but huge breakthroughs and really fast.
  
• Multicentric world - if we had a world government, we could just take a vote, set up whichever emission caps or taxes we wished, problem solved. If we had a small number of important powers like we do now, we could get them to agree on some flawed solution. But the world seems to be evolving towards multicentricity - right now only US, EU, and China matter, plus maybe Japan, India, and Russia a bit - but in the future other countries will probably count a lot too - and the more players you have, the harder it will be to reach meaningful agreement.
• Economic development of rich countries - CO2 emissions per capita don't increase significantly past certain level of income, so it won't matter much either way.
  
• Demographic growth - faster growth means more emissions, but uncertainty about 2050 population isn't that high, so it can only have modest effect.
• Attitude towards nuclear energy - if people become more supportive of nuclear energy, it will reduce CO2 emissions somewhat.
• Policy of US, China, India, and the entire developing world - matters a lot, as they are current and future main emitters, they have a lot of demographic and/or economic growth ahead, and it's highly uncertain what their policies will be.
• Policy of EU - matters very little, as EU is already highly efficient for GDP it has, has little economic or demographic growth, and it can be relied on to do further reductions no matter what others will do.
  

Politics

And regardless of those, the most important factor is politics - global warming is a very unusual situation in which not only everybody's emissions affect everybody equally (straightforward externality / tragedy of commons), but also everybody's current emissions only affect people living in far future. We're asking for 2009's Americans to make significant sacrifices for the sake of 2100's Bangladeshis. 3000's Bangladeshis even, as IPCC says that "anthropogenic carbon dioxide emissions will continue to contribute to warming and sea level rise for more than a millennium".

Let's take an outside view, and make a reference class forecast - in situations in which current sacrifices of people living in one country could make a difference to people living in entirely unrelated country a century ahead - how often were the sacrifices made? To the best of my knowledge the answer is a reliable never. It just doesn't happen. If one country like EU now (and to not that significant extent) were to sacrifice money to limit its emissions, others would just take advantage of cheaper fossil fuels due to reduced demand and burn more of it.


In theory a global carbon emission market could be established, but good luck getting everyone to agree on allocation of emission rights, or income from their sales. The more severe the cuts, the more expensive will emission rights get, and the worse will the fight be. If we allocate most rights to rich countries, poor countries will correctly complain that it hinders their economic development. If we allocate most rights to poor countries, it will amount to massive transfer of wealth from the rich to the poor, and good luck getting voters to agree to that.

We're talking about tens of trillions of dollars. It's not a metaphor, just the proposed American system is estimated to be valued $80bln / year - $80bln times 40 years and 5x (world emissions / American emissions), it will be worth $16 trillion, and growing as supply gets restricted and demand grows due to economic development. How will these tens of trillions of dollars in either emission rights or auction profit receipts be allocated? Yes, good luck with that.

The alternative is not caring about greenhouse gases emissions, and geo-engineering the temperature growth away if need be.

What is libertarianism - blog edition

Usually when people talk about politics, they use language of good and evil. So if you ask an American "liberal", "conservatives" are stupid, bigoted shills for the super-rich willing to destroy the planet and kill people home and abroad in name of some short term profits.

And if you ask an American "conservative", "liberals" are baby-murderers who destroy economy by excessive taxation, overregulation, useless spending, and huge budget deficits.

Some of these accusations are valid, some are nonsense, in many cases politicians from both sides are equally guilty. And it's not a uniquely American phenomenon - in most countries people's levels of rationality reach their lows when discussion turn to politics. This isn't really intellectually satisfying.

Another way to talk politics is what Tyler Cowen's has done in his series of posts about progressivism, conservatism, transhumanism, and then some others like Leigh Caldwell's about libertarianism etc. The idea is to describe a political view as a list of beliefs, about which all sides can agree that Xers believe in them and non-Xers don't, without taking sides on their truth-value.

One problem with this approach is that it has no way of distinguishing core values of an ideology from insignificant baggage that grew around it due to historical processes. Is opposition to gay marriage central to conservatism or fairly irrelevant? How important is public option to liberals? These are interesting questions.

Blog edition

So I wanted to try something else - figuring out what some political ideology is based on what its adherents blog about. It is in a way a fairly objective standard - assuming we can figure out a list of top blogs from some perspective, and that we can categorize posts' subjects in a unbiased way, every analysis should reach similar conclusions regardless of analyst's views.

I took the following list of "Top 20 Libertarian blogs", with top 5 posts ignoring extremely short ones which just link somewhere else, and other funny/unrelated posts. List is very UK-centric, but it doesn't matter.

1. Guido Fawkes
   
2. Devil's Kitchen
3. Old Holborn
4. Obnoxio the Clown
5. Underdogs Bite Upwards
6. Tim Worstall
7. Samizdata
8. Boatang & Demetriou
9. Dick Puddlecote
10. LPUK Blog
11. Last Ditch
12. Constantly Furious
13. Anna Raccoon
14. Freedom to Choose
    
15. Rantin' Rab
16. Plato Says
17. Charles Crawford
18. An Englishman's Castle
19. Frank Davis
20. Oxford Libertarian Society

And the winner is...

I'm actually surprised by the most popular subject - climate change denial. 20% of posts (++++++++++++++++++++) deal either deny climate change or are skeptical about it, which in most cases just denial without balls. Further 12% (++++++++++++) are about Copenhagen summit or environment without being obviously denialist. It's really surprising, as global cap-and-trade system would be the best test case of how well libertarian / free market solutions can deal with problems of externalities, so naively libertarians should be extremely enthusiastic about it - and in any case there's no obvious reason why political ideology should have so much influence over question which is essentially apolitical - temperature changes.

Shilling for the superrich

The second most popular subject doesn't surprise - it's good old shilling for the rich. 10% of posts (++++++++++) were against taxing the rich, or regulating salaries and bonuses in taxpayer-bailed-out banks. Because owners' right to regulate management salaries ends the moment the owner is government. Related 4% of posts (++++) are generic anti-tax posts - but it's quite clear which taxes anger libertarians the most.

A surprise third most popular subject...

I sort of expected climate change to be popular, as Climategate and Copenhagen summit are in the news right now, but this one baffled me. It turns out the third most popular subject with 9% of posts (+++++++++) is smoking - with posts either opposing public places smoking bans (what about "rights" of non-smokers not to have smoke blown in their faces?), or even expressing skepticism about links between smoking and cancer.

The long tail

Further categories don't suprise. 5% (+++++) are offensive posts without much content - something you can expect in political blogs. 4% (++++) are anti-EU rants, 3% are (+++) culture wars / political correctness rants, further 3% (+++) are about budget deficits; copyright (++), religion (++), new media (++), and UK politics (++) get 2% each. There was a single post about each of: discrimination in Iraq (+), gold standard (+), Twitter (+), modern art (+), quality of mainstream media (+), regulation of banking (+), NHS (+), science (+), UK voting by mail (+), forced labour (+), Gordon Brown's typos (+), privacy (+), family law (+) natural rights (+), anarchy (+) Kosovo (+), and public funding of festivals (+) - all expressing the positions you'd expect. There was one pro-Iraq war post (+), one against ideology in politics (+) - notice the irony, and one posts each about Tiger Woods (+), Sarah Palin (+), and Ludwig von Mises (+). The libertarians' favourity shitty book writer Ayn Rand is strangely missing from the list.

Summary

I read a lot of blogs with which I disagree, but they tend to be high quality and quite insightful. So my contempt for libertarianism doesn't automatically mean I wouldn't find at least some good libertarian posts - Tyler Cowen and Robin Hanson are fairly libertarian, and they're both in my RSS. But the posts I've seen were just so bad that of 100 I've checked I cannot point a single one that had any new insights or was interesting in any way. Few even pass basic sanity tests - not just by being contrarian - contrarian posts are much more interesting to read than ones that repeat the conventional wisdom - but by simply not having any idea what they write about.

Why CO2 emissions will not fall before 2050

In my previous posts I explained why emissions are unlikely to fall below current levels before 2050, and why due to peak heating occurring long after peak emissions we do have a very long term problem. Conclusion being - geoengineering is pretty much inevitable.

In this post I'd like to talk about two things - what the world will look like in 2050, and the Climategate.

First, the Climategate. As shown by the leak, scientists from East Anglia University engaged in some highly unethical practice, such as knowingly fudging data to support their thesis, violating FOIA and hiding data to the point of deleting it to avoid release, and organized bullying campaigns against scientists with the opposite views. This makes them bad persons, and if God existed, they'd suffer in Purgatory for it / or got shitty reincarnation in the next life / or something like that. Fortunately for them there is no God, so they will get off scot-free with it.

But this behaviour, while highly unethical, doesn't really seem to change the main conclusion of IPCC reports that anthropogenic releases of greenhouse gases have caused significant warming, and are likely to cause a lot more in the future.

The only consequences for climate predictions I can think of is that it might damage people's opinion of credibility of climate change science disproportionally to the breach of proper scientific practice (which did occur, but were fairly minor), thus reducing political will to act on climate change, and increasing the problem. But then, I doubt the damage will be that big.


Now to the main subject - world in 2050. There's no question that we'll stop emitting ever increasing amounts of greenhouse gases at some point. We already have at least four ways to entirely solve the problem even without any significant technological breakthroughs: nuclear, solar thermal, solar photovoltaic, and wind. But the question of when is a big one, and I doubt it will happen soon.


As I wrote in previous blog posts, in 2050 population will be around 9-10 billion, per capita GDP will be around current EU levels, and so will per capita energy use. But energy use and carbon emissions are not that closely connected - there are gains to be made in efficiency, both nuclear and renewables emit little in terms of greenhouse gases, so that doesn't directly imply much about per capita CO2 emissions (nor emissions of other greenhouse gases).

Let's make a simple table - what would have to change for 2050's CO2 emissions to be equal to 2009's CO2 emissions. For simplicity let's assume every country's per capita emissions will be the same for now. If some countries don't take their share of the burden, others will have to.

Country	Population (mln)			CO2 emissions (mln ton)			CO2 emissions per capita (ton/person)
	2009	2050	Change	2009	2050	Change	2009	2050	Change
China	1334 [19.6%]	1462 [15.7%]	+9.6%	6103 [21.5%]	4458 [15.7%]	-27.0%	4.57	3.05	-33.3%
USA	308 [4.5%]	397 [4.3%]	+28.9%	5752 [20.2%]	1210 [4.3%]	-79.0%	18.68	3.05	-83.7%
EU-27	499 [7.3%]	424 [4.5%]	-15.0%	3914 [13.8%]	1293 [4.5%]	-67.0%	7.84	3.05	-61.1%
Russia	141 [2.1%]	104 [1.1%]	-26.2%	1564 [5.5%]	317 [1.1%]	-79.7%	11.09	3.05	-72.5%
India	1173 [17.3%]	1572 [16.9%]	+34.0%	1510 [5.3%]	4794 [16.9%]	+217.5%	1.29	3.05	+136.9%
Japan	127 [1.9%]	109 [1.2%]	-14.2%	1293 [4.5%]	332 [1.2%]	-74.3%	10.18	3.05	-70.1%
Other countries	3214 [47.3%]	5253 [56.4%]	+63.4%	8293 [29.2%]	16021 [56.4%]	+93.2%	2.58	3.05	+18.2%
World	6799 [100.0%]	9322 [100.0%]	+37.1%	28431 [100.0%]	28431 [100.0%]	+0.0%	4.18	3.05	-27.1%

Data sources: emissions, population projections. EU-27 is used as European Union will most likely expand before 2050, and I don't want considerations like that to complicate matters.

Right now just six big entities - China, India, USA, EU-27, Russia, and Japan - emit over 70% of global CO2. But poor countries will develop, and this means more emissions. India's emissions are ridiculously low now - due to poverty - but once Indians get richer they will want cars. They will want pretty things. This will use energy, and this will emit CO2. The same story will take place in the "rest of the world" category (which unfortunately mixes small rich countries with big poor countries, obscuring the change).

What kind of emission cuts will the big emitters have to do just to counteract population growth of the world and economic growth of poor countries? Assuming the burden will be equally shared, USA will have to cut by 83.7%, or to 1/6th of current per capita levels. EU, Japan, and Russia will have to cut by about 2/3 per capita, and even China will have to cut by 1/3 per capita, in spite of so much economic development still being ahead of them! Other countries will have a mix of increases for the poor, and very significant decreases for the rich.

Now how likely is this scenario? I quite doubt we'll actually achieve equal sharing of burden.  Cuts for developed countries people talk about tend to be around 15% levels by 2025, so presumably around 40% by 2050, and even these are best case scenarios.

Intrade thinks that odds of any cap and trade system being established in USA by 2011 is about 50:50, and that says nothing about levels of caps, and severity of loopholes passed due to industrial lobbyists' money. After that it's Palin's administration as Krugman says, so don't hope for any meaningful limits. In general, people vote for current party when economy is good, and vote against current party when economy is bad, no matter who's guilty. Current predictions of American unemployment for 2012 are so high, that Democrats are very likely to be voted out of power unless they get their act together.


Anyway, 40% by 2050 is nowhere near enough to even keep 2050's emissions equal to 2009's! So rich countries, especially USA, are not providing the leadership here. What are the chances that poor countries will voluntarily take most of the burden of reducing carbon emissions all on their own? I'd say extremely slim.

And, most importantly... even if reductions from the table did happen, and 2050's emissions were at 2009 levels and falling, this still leaves us with massive global warming problem as peak warming happens long after peak concentrations, and peak concentrations happen long after peak emissions.

What brings us to where we started - there's simply no realistic scenario under which geoengineering can be avoided.

Medieval 2 Total War Concentrated Vanilla 0.08

Time for another version of my Medieval 2 Total War minimod. It doesn't contain that many changes over 0.07.

The biggest change is fixing the "everyone hates you" bug I wrote about a few days ago. That doesn't mean everyone will love you now - but now you have the option of acting chivalrous, getting good reputation, and if you do your alliances will be more likely to survive, your enemies will be more likely to sign peace with you instead of insisting on silly wars until you destroy them and so on. The game still makes everyone hate you if you're too strong, or if you're playing on very hard difficulty, so don't worry about it being too easy.

Rubber swords bug

The second bug fixed is the extremely well known "rubber swords" bug. In Medieval 2 Total War units have between 1 and 3 weapons, usually 2. It's absolutely necessary for missile units, which need some sort of missile weapon, and also some secondary melee weapon in case they have to fight man to man. But many units have two melee weapons - most knights have a lance for charging and sword for close combat.

Like many things this all makes sense in theory. The problem is with pikemen. They have long pikes for fight in formation, and swords for close combat. Unfortunately their logic for weapon choice is pretty much the same as with knights and their lances - instead of trying to use pikes as long as they can, they switch to swords almost immediately after first charge. And with very weak swords, no shields, and weak armor pikemen make for really horrible infantry.

If it was just a matter of them being weak, it wouldn't be that big of a deal - there's plenty of weak units in game. The real problem is that they switch to rubber swords too early, and you don't see any pike action at all! Removing their "rubber swords" completely makes them much stronger, and behave properly.

Now there's a valid counterargument, that is raised on Total War forums every time the rubber sword bug is mentioned - the game was balanced with rubber sword behaviour, Pikemen are extremely cheap, and so shouldn't be expected to be much good.


It has some validity - Pike Militia costs only 150 florins, half as much as Town Militia at 290, so making them into some sort of super-unit would unbalance the game. Recruitment costs are only meaningful in battle mode (that is mostly multiplayer). In campaign mode, cost of an unit is cost of its recruitment, upkeep over multiple turns, and of buildings necessary to recruit the unit.

Compare scenarios of building required buildings (other than basic things like walls and barracks up to city watch and you need anyway), recruiting 10 (for vanilla, or 15 for mod as buildings are 50% more expensive) units from it, keeping them for 5 turns in garrison, and 5 turns in field. Costs per unit would be:

• Town Militia: 290 recruitment + 0 garrisoned + 625 in field + 0 only basic buildings = 915
• Genoese Crossbow Militia: 570 recruitment + 0 garrisoned + 500 in field + 0 only basic buildings = 1070
  
• Peasants: 110 +  450 garrisoned + 450 in field + 60 mustering hall = 1070
  
• Italian Spear Militia: 460 recruitment + 0 garrisoned + 625 in field + 0 only basic buildings = 1085
  
• Peasant Archers: 220 recruitment + 500 garrisoned + 500 in field + 120 bowyer = 1340
• Peasant Crossbowmen: 220 recruitment + 500 garrisoned + 500 in field + 360 practice range = 1580
  
• Pike Militia: 150 recruitment + 0 garrisoned + 625 in field + 960 militia barracks = 1735
• Mailed Knights: 680 recruitment + 1250 garrisoned + 1250 in field + 0 only basic buildings = 3180
• Lancers: 930 recruitment + 1600 garrisoned + 1600 in field + 2760 all stables = 6890
  

Recruitment costs are simply a tiny fraction of total costs! Now this depends on which building you'd build anyway (you always need walls; city watches increase public order, shipyards increase trade etc. so you might buy them anyway), how many units you're going to recruit from each such building, how long you're going to keep them garrisoned, and how long in field and so on. These are not definite numbers - but they should give you some idea. It's quite easy to spot the most overpowered and most underpowered unit on the list here - Genoese Crossbow Militia is as cheap as Peasants, but they're ridiculously stronger - it's one of the best missile units in game versus absolutely the worst unit in game!

So anyway, I did some experimenting with pikemen. I've heard about two solutions on Total War forums - either remove their secondary weapons, or make their formations tighter. I made two AIs fight each other - 3 Pike Militia versus 3 Italian Speak Militia. Results were:

• Normal Pike Militia - lost, 86% dead vs 37% dead
• Tight formation - lost, 83% dead vs 58% dead
• No secondary weapons - won, 66% dead vs 80% dead
• Tight formation + no secondary weapons - lost, 88% dead vs 88% dead
  

The only reason tighter formations work is by making it harder for the enemy to get real close, and making pikemen use their pikes longer. Now AI was clearly unbelievably dumb here - happily charging on the pike wall. Manual results were:

• Me normal Pike Militia, AI Italian Speak Militia - I won, 65% dead vs 89% dead
• Me Pike Militia without secondary weapons, AI Italian Spear Militia - I won, 37% dead vs 85% dead
• Me Italian Spear Militia, AI Normal Pike Militia - I won, 1% dead vs 20% dead
• Me Italian Spear Militia, AI Pike Militia without secondary weapons - I won, 4% dead vs 13% dead
  

So properly used, Italian Spear Militia is far stronger than Pike Militia - properly used means flanking, and not blindly charging onto the pike wall. Nothing too complicated. Claims of Pikemen being overpowered without secondary weapons seem greatly overstated.

Scaling down previous changes

I decided to scale down a few changes. Cavalry no longer costs 50% extra - nerfing them was fun, but now that most infantry gets free upkeep in castles, and pikemen actually work, cavalry is decently nerfed anyway.

I reduced extra ammo for missile infantry from +100% to +50%. It doesn't really make that much difference, and I'd rather stay closer to vanilla values unless I have a good reason not to.

Full list of changes

Here's the full list for reference

• Fixed standing bug
• Fixed rubber swords bug
• Castle garrisons free for basic infantry
  
• City garrisons twice as big
• King's purse x2
• Rebel spawn rates 10x lower (supposedly, I have my doubts if it works)
• Missile infantry ammo +50%
• Bodyguard unit sizes halved
• Building construction times all 1 turn
• Mining income x2
• Building costs other than mines +50%, mines x3
• When defending a settlement, towers are activated from much greater distance
• Wall and gate strength x5, but tower strength unchanged
• Tower fire rate x2, but only for non-flaming ammo
• Spy costs x2
• Campaign movement speed +75%

Here's the download.

Dietary consequences

This post is a follow-up to The myth of 5 a day - people never ate that much fruit and veg.

"Because" is not a logical connective of classical logic, it's not possible to construct a truth table for "A because of B" statement. Even when A and B are true, it's really quite difficult to prove that causality flows from B to A - correlations, no matter how strong, don't prove causality.

On the other hand, while proving such statement true is hard, proving it false can be very simple - if either A or B is false, "A because of B" must necessarily be so too. So "crime is on the rise because of violent video games" can easily proven to be false, because the consequence part ("crime is on the rise") is false - therefore collapsing the whole argument before we get anywhere near analyzing causality.

"Guns prevent crime" / "death penalty prevents crime" arguments collapse equally fast - countries with more guns and more executions don't have less crime - so we don't even need to think much about causality here. That doesn't mean that reverse statement "gun control prevents crime" is necessarily true, causality might be missing even if correlation exists. By the way there is actually some pretty good research indicating that violent movies deters violent crime, at least in short term - apparently because violence-loving young males go to theaters to watch violent movies instead of getting into real crimes. It's not too unlikely that it might be the same with violent video games.

Now that we're past the obligatory cat and the obligatory detour, let's get back to diets. There's a widely repeated claim, which basically goes "people are fatter than in the past because their diets have much more fat, especially animal/saturated fat than in the past, and nowhere near as much fruit and vegetables as in the past". The consequence is undoubtedly true - obesity is on the rise pretty much everywhere.

On the other hand, the part about too much (saturated/animal) fat, and not enough fruit and veg is entirely wrong. We actually eat a lot less fat; the fat we eat is increasingly unsaturated vegetable fat; and we eat a lot more fruit and vegetables than in the past.

Take a guess - what are the biggest sources of calories in American diet? Here's the list for 1961 and 2003, in kcal/day. I don't believe 1961's diets were perfect, and I'd love to have data earlier than 1961, but that's as far back as FAO statistics go.


Vegetal products consumption increased a lot - 1871 (65%) to 2708 (72%) kcal/day, while animal products consumption stayed constant in absolute terms, and decreased a lot proportionally - 1010 (35%) to 1045 (28%).

Top vegetal products:

• cereals without beer - 627 to 832 (of which wheat - 504 to 603; rice 26 to 94)
  
• sugar and sweeteners - 515 to 657 (of which sugar 453 to 320; HFCS 56 to 331)
  
• vegetable oil - 276 to 606 (of which soybean oil 157 to 492)
  
• alcoholic beverages - 108 to 150 (of which beer 68 to 100)
  
• "fruits" without wine - 79 to 117 (of which oranges 14 to 32)
  
• potatoes - 77 to 100
• maize (without HFCS, maize oil etc.) - 59 to 98
  
• "vegetables" - 63 to 77
• beans - 32 to 30
  

It's quite surprising list - foods which can be considered traditional like wheat, potatoes, and beans, are a really small portion of the whole. Modern vegetal products like soybean oil (unsaturated fat), sugar+HFCS (sugar, obviously not fat) dominate the list.

And for animal products:

• meat - 335 to 451 (of which poultry 64 to 197; pork 127 to 132; beef 125 to 115)
  
• whole milk - 304 to 199
• animal fat products (butter etc.) - 199 to 116
• cheese - 48 to 149
• eggs - 67 to 55
• butter - 65 to 40
  
• fish and seafood - 19 to 28

Total fat and protein consumption in g/day:

• Animal fat - 69g to 71g (63% to 46%)
  
• Vegetable fat - 40g to 83g (37% to 63%)
• Total fat - 110g to 155g
  

What increased here was poultry - the approved lean meat; and cheese - the most vegetarian of animal products; what decreased most was whole milk, butter, eggs, and animal fats as whole category.

This is as far away from the conventional story as it gets. Proportionally to their diets, people eat less fat, a lot less of the supposedly "unhealthy" saturated animal fat, less milk and butter (saturated fat), vastly more "healthy" vegetable fat, more lean meat, more vegetables, more fruit. Sugar+HFCS consumption increased a lot in absolute terms, but stayed fairly constant proportionally, even decreasing slightly from 17.8% to 17.5% of all calories.

So it's not that people lack personal responsibility - people adjusted their diets to follow every single point of conventional dieting advice. To keep blaming people's eating habits for obesity epidemic is to blame the victim. You cannot claim that people are fat because they don't follow dietary advice, as they actually do!

The only alternative is that the conventional advice doesn't work. Eating more fruit and veg, switching from animal to vegetable fats, reducing proportion of fats in diet and so on - does not make you slimmer. I won't answer the question if the advice is merely useless or actually harmful in this post (causality in reverse direction) - that would require a lot of research. But we sure know it doesn't work.

I doubt that another round of minor adjustments to the advice (look at all reversals on butter vs margarine), let's say by insisting on more fish oil this time, are likely to magically make it work now, in spite of consistent track record of failure.

I don't have enough data to give you reliable advice, but the most obvious alternative would be eating reasonably-sized portions of the most delicious food you can think of. This will usually be something fairly traditional (even not necessarily from your culture), often but not always with plenty of animal products in it, and extremely rarely with much sugar or vegetable oil. If type of food doesn't matter, and only portion size does, then you can as well enjoy it, instead of torturing yourself with diets.

For the record, author of this post has BMI 23. Never trust overweight people to give you diet advice.

Why everyone hates you in Medieval 2 Total War

Short answer - because of a really stupid bug in data files. If you want it fixed, download corrected file and put it in your C:\Program Files\SEGA\Medieval II Total War\data\ directory or similar. For the long version, read on.

Reputation and standing

First, technicalities. The game keeps track of two important numbers - your reputation (also called global standing), which is global, and standing with a particular faction. Both are numbers between -1 (the worst possible) and +1 (the best possible).

Good behaviour increases your reputation, and your standing; while bad behaviour decreases it - as expected. There are two kinds of deeds - some actions are unconditionally good or bad - they simply add or subtract from your reputation/standing. If you do 5 good deeds and 4 bad deeds of this kind, it's pretty much the same as just doing 1 good deed.

The other kind use what the game calls "normalization". This "normalization" operation has two parameters - target value, and "divisor". It takes your current standing or reputation, and moves it by 1/divisor towards the target.

For example if your current standing is 0.6, then "normalizing towards +1 with divisor 10" will result in standing of 0.6 + (+1 - 0.6)/10, that is 0.64. If your initial standing was -0.8 instead, the result would be -0.8 + (+1 + 0.8)/10, or -0.62. In other words - if your reputation/standing is bad, good deeds improve it a lot, while bad deeds damage it little. And if your reputation/standing is good, good deeds improve it little, but bad deeds damage it a lot. In our example, the good deed had value of +0.04 for the well-reputed faction, but +0.18 for the ill-reputed one.

Normalization to some value with divisor 1 is equivalent to completely changing your reputation/standing - by our equation initial + (target - initial) / 1 = target. Doing both good and bad things of this kind will move your reputation towards neutral values around 0.0, even if one kind of deeds is more frequent than the other. It's all quite sensible in practice.

Obviously we want our relations with different factions to be high - factions that hate us will be more likely to start wars against us, disrespect treaties and so on. Even with enemies we would prefer higher standing, in case we want to sign peace treaty someday.

What changes reputation

Most reputation changes are absolute.

First, your reputation normalizes to 0 with divisor of 200 anyway, which is extremely slow - it takes 140 turns to halve your good or bad reputation. Having an ally normalizes your reputation to +1 with divisor of 400, and being at war (except with rebels obviously) normalizes your reputation to -1 with divisor of 800. These are extremely small divisors, but if you have multiple allies, or multiple enemies they add up.

On top of that, some generic good and bad deeds:

• "Taking back" settlement, which you gave by diplomacy (but then do you ever give settlements by diplomacy?): -1.0
• Attacking an ally: -0.5
• Canceling alliance (also when forced by one of your allies attacking another - tough luck): -0.1
• Bribing settlement (it's virtually impossible in Medieval 2 Total War anyway, unlike in Rome Total War where you would be doing it all the time): -0.15
• Breaking treaty terms (I'm not sure what counts here): -0.15
• Military assistance to your ally in battle: +0.1

Treatment of prisoners is probably the most important source of reputation. If you put your prisoners up for ransom, it doesn't affect your reputation either way - this is the standard expected behaviour in Medieval warfare. What changes it is either releasing or executing prisoners.

• Releasing prisoners: +0.01; +0.025 for groups of over 80 soldiers; +0.025 if any family member was released.
• Executing prisoners: -0.01; -0.025 for groups of over 80 soldiers; -0.025 if any family member was executed.

I'll get back to this later in more detail.

And here we arrive at the bug. Treatment of civilians is supposed to affect your reputation:

• Occupying a settlement peacefully: +0.05
• Sacking a settlement: -0.02
• Exterminating population: -0.05
  

But due to a bug the first never happens. Here's the data file:

;------------------------------------------
Trigger 0103_occupy_settlement_increase_global
WhenToTest OccupySettlement

FactionStanding global 0.05
FactionStanding target_faction normalise 1.0 20
FactionStanding target_allies normalise 1.0 40
; FactionStanding target_enemies normalise -1.0 40

;------------------------------------------
;Trigger 0102_city_razed_decrease_global
; WhenToTest CitySacked

FactionStanding global -0.05
FactionStanding target_faction normalise -1.0 10
FactionStanding target_allies normalise -1.0 20
; FactionStanding target_enemies normalise 1.0 20

Semicolon means code is commented out and inactive. The first event describes what happens when settlement is peacefully occupied. The second is old sacking code, probably from Rome Total War times. It's supposed to be commented out, but only trigger was commented out, the effects were not. So the results are equivalent to:

;------------------------------------------
Trigger 0103_occupy_settlement_increase_global
WhenToTest OccupySettlement

FactionStanding target_faction normalise 1.0 20
FactionStanding target_faction normalise -1.0 10
FactionStanding target_allies normalise 1.0 40
FactionStanding target_allies normalise -1.0 20

So not only you don't get the promised reputation bonus, your standing with the target and their allies gets lower, instead of higher - they're actually happier when you sack their settlements than when you occupy them peacefully, what is nonsense! By the way, all websites about Medieval 2 Total War I've seen seem unaware of the bug, even though it's trivial to test.

Just see how different the game is with and without bug. If you take 6 neutral settlements, your reputation should rise to 0.3 (Reliable); with the bug it will stay at 0.0 (Mixed). This has huge effect on how others will treat you!

Standing

Now that we know about your reputation, how about standing? There's extremely long list of rules related to the Pope, crusades and such, which I'll cover later. For normal factions it's much simpler. I'll spare you the numbers, but basically:

• Factions with high reputation get liked more, factions with low reputation get liked less. Thresholds are -0.4, -0.1, +0.1, +0.4.
  
• Depending on difficulty, your standing tends to drift towards +1.0 (very easy), 0.0 (medium), -0.5 (hard), and -1.0 fast (very hard).
• Factions which are strong get liked less, factions which are weak get liked more.
• There's small advantage to having the same religion.
  
• Your allies grow to like you more, and your enemies to like you less. Allies/enemies of your allies/enemies change their standing accordingly, by smaller factors. If this works correctly, you can see two big coalitions developing, and it often happens.

There are miscellaneous actions that increase or decrease your standing. In addition to the obvious acts of war, bribing, sabotage, and spying decreases your standing - spying much more than I thought - to -1.0 with divisor 20, and with divisor 80 for all allies. I'm not sure if it counts only when spy is caught, but it's far higher than I expected. Only 13 acts of spying will move perfect standing all the way to neutral.

What doesn't affect your standing is merchants buying each other out, your princesses marrying (and stealing) their generals, and your priests preaching on their territory. Marrying your princess to their faction heir is worth a lot - +0.2 instantly, and normalization to +1 with divisor 20 every turn.

Adding it all together - the way it's supposed to work is that increased standing due to high reputation should counteract decreased standing due to your size and difficulty level (at least on hard, maybe not so much on very hard). But because you don't get reputation points you deserve for peacefully occupying settlements, there's no balance, and everyone gets to hate you.

The Pope

It's only vaguely related to the bug this post is about, but as I started talking about Medieval 2 Total War standing system, I can as well describe how it works with the pope.

First you get points for building religious building - from 0.02 for small churches and chapels, to 0.2 for huge cathedrals, and additional 0.2 for your first huge cathedral, and additional 0.2 if it's the first huge cathedral in the world - so you can get +0.6 with the pope for just one building, after first getting +0.52 for first cathedral in the world.

On the other hand destroying religious building for money makes the pope hate you, about 5x as much as building them. But as you never really destroy buildings anyway, you probably don't need to bother yourself with it too much.

Crusade get you huge bonuses with the Pope - but only arriving at target region, and taking the target settlement, not just joining. You get bonuses for every general, and every unit, but a lot more if you send your heir (+0.2) or your leader (+0.6) to the crusade.

Something I didn't know about before reading the code is that attacking Orthodox factions while on crusade (usually taking Constantinople on the way to Jerusalem or Antioch) gets pretty high penalties with the Pope. You'll get your papal standing back if you actually conquer your target, but if the crusade is just an excuse for sacking the Byzantine Empire the Pope will not be happy. As you probably already know, attacking non-excommunicated Catholic factions while on Crusade, or Islamic factions while on Jihad is prohibited by the rules.

Recruiting priests and doing papal missions get you points. Getting excommunicated loses you -1.0 points, obviously. Getting your cardinals promoted get you points (0.1 per cardinal), and there are even more points in elections - Pope from your country means +0.8, from allied country means +0.4, enemy country's Pope means -0.6, and voting for the winner pope is +0.2. Surprisingly I don't see any negative points for voting for the wrong candidate, even though in-game text insist on wrong vote being a major transgression.

You also lose points with the Pope for being at war with other Catholics. And look after heretics in your lands - if the Pope has to appoint an Inquisitor, it costs you immediate -0.2.

Now it's going to more or less even out - you will build churches and recruit priests anyway, especially if you're close to Islamic or Orthodox regions - what will score you major Pope points. And you'll lose points for inevitable wars with other Catholics. You're supposed to get extra points for good reputation, but as I said, you won't due to the bug.

What to do with prisoners

There's one more thing I wanted to write about - treatment of prisoners. You naturally know that you shouldn't normally sack or massacre settlements, as these will be your subjects, but why do anything except execution with the prisoners? Let's do some game theory:

• Value of reputation gained or lost: A (by marginal view it's supposed to be the same either way, it's not exactly true, but let's keep it this way for simplicity)
• Value of money gained from ransom: B
• Cost of enemy getting units back: C
  

So payoffs are:

• Released: A-C
• Ransom accepted: B-C
  
• Ransom rejected: 0
• Executed: -A
  

It helps to know if the enemy is likely to accept the ransom or not. Usually the answer is clear - due to war they don't have any money, so they cannot pay you. In this case if A>C, you release prisoners, otherwise you ask for ransom and don't get it. A>C is the usual situation, you've just defeated their armies, so they probably weren't that good anyway. The biggest exception I can think of is when you're on your way to take a weakly defended settlement, and these armies would join the defenders and make your work much more difficult when released. Or when it's really late in game, you're at war with everyone, and you don't care about reputation anyway.

How about when you think the ransom will be accepted? First, decision between release and ransom. If A-C>B-C, release straight away. Unless you captured their leader (10000) or heir (5000), reputation will be almost always more valuable than money - mostly because for reputation purposes they're treated just like any other general, but for ransom purposes they're a lot more valuable. Except your enemies are quite unlikely to have that much money, so it's not a common occurrence.

How about executing them? It only makes sense when C>2A, and C>A+B. Reputation cost of execution is twice as high, as now you have to actually execute the prisoners, instead of just request ransom. Your loss due to release of troops is very rarely that high. Even tiny group of soldiers released or executed will make a difference of 0.02 reputation points, about as much as you need to counteract being at war with someone for 16 turns. If you act chivalrously wars improve your reputation instead of damaging them. Paradoxically the worst kind of war for your reputation is a prolonged war in which nothing happens.

This is all a lot less true once the bug is fixed. Due to the bug releasing prisoners is the only way to significantly improve your reputation, and you need that just to stay positive, as you'll lose it due to broken alliances, executions of prisoners, and sackings of settlements that will inevitably happen sometime during the game. Having allies and assisting them militarily just won't be enough - once big major start, getting allies will get really tough, especially if your reputation isn't very high (chicken and egg...), and assisting stupid bots requires way too much of effort.

But without the bug, you should have decent reputation from just straightforward conquest, unless you kill and sack on your way - and value of A gets a lot lower.

To fix or not to fix

With minor bugs the question is clear - definitely fix them. But this bug is so big, and changes the game so much... Not only AI will like you more, AI will like each other more too. And acting chivalrous will be much more important - with buggy M2TW you really had to go out of your way to get decent reputation, but with the fix if you behave you'll reap diplomatic rewards.

Most likely the fix will make the game significantly easier - as long as you act chivalrous it will delay your wars and make your enemies much more likely to agree to cease fire. On the other hand it will make AI factions hate each other less too - so who knows, maybe they'll unite against you? At some point you'll be big enough to overwhelm your reputation-based bonus, so maybe it doesn't matter that much...

I'll try my next game with the fix, and tell you how it went. It's a really long post for a bug that's basically 3 missing semicolons.