The best kittens, technology, and video games blog in the world.

Thursday, November 05, 2009

Inevitability of geoengineering

Cat and fire by Michel Filion from flickr (CC-BY)
I want to write a quick post showing some arguments about inevitability of large scale geoengineering which I find extremely convincing. I'm sure I won't convince that many people, most have made up their minds already.

First, look at list of countries by CO2 emissions. In 2006 global emissions were 28.43 billion tons per 6.55 billion people, or 4.34 ton/person. World population is still growing and is estimated to stabilize at levels around 9.5 billion for 2050-2100, which is the time in question.

Now let's see what would happen if everyone got to current European standards of living, and European CO2 emissions, which are 7.84 tons/person. This would result in CO2 emissions of 74.5 billion tons, or 2.6x times current levels. This number is already quite optimistic - it makes a huge assumption that all countries with higher CO2 emissions like US, Japan, Canada, Australia, Russia etc. scale them down quite considerably.

Now a brief discussion is in order on why numbers like that are relevant. First, I assume that there won't be a massive genocide, or nuclear warfare, or pandemic, or anything else that could drastically reduce population levels. The last time there was a significant reduction in population levels was 1340s. None of the world wars, or any other events even stopped population growth - in fact people tend to reproduce a lot more during times of crisis, than during times of peace and prosperity, so it would need to be the biggest war, pandemic, or other kind of failure in history to significantly reduce population levels. This assumption is probably the safest.

The second assumption is that the rest of the world will grow quite quickly. World GDP/capita is currently $9,774 PPP. EU GDP per capita is $30,513 PPP. If world economy grows at per capita rate of 2.8%, then by 2050 world average GDP will reach current EU levels. This coincidentally is almost exactly annual per capita growth rate of the world economy of the last 30 years. Unlike with population, where predictions are quite solid, here there are more reasons why growth could be significantly slower, or significantly faster than predicted. We just had a completely unexpected global recession for a fairly obvious example. And not so long ago we had massive completely unexpected boom in China and then India, so it can go both unexpectedly badly and unexpectedly well. Still, 2.8% is a pretty decent first estimate.

I want to write a bit more on the subject of economy. Many people are sceptical about the idea that the "poor" countries will ever attain levels of economic development seen in rich countries. But the rich-poor divide is illusory. As you can see yourself with Gapminder, the divide is long gone and most countries are in the middle. World average life expectancy today is higher than in most richest countries in 1950s years ago. World average life expectancy in all but one countries is higher than world average 100 years ago - all that in spite of wars, malnutrition, malaria, HIV, and lack of clean water in many of them! Life in countries considered by many to be "permanently poor" is vastly better than it was in Europe of your grandfathers, and rapidly improving. To think it will somehow suddenly stop is to ignore history.

Development can happen extremely rapidly - world's largest CO2 emitter has been China for a few years now, per capita emissions of which increased by factor of x2.2 between 1990-2006. Likewise for India, growth was x1.6 during that time. Even for big countries not typically associated with rapid growth you have high growth - x4 for Vietnam, x2.4 for Thailand, x2.3 for Malaysia, x1.9 for Indonesia, x1.5 for Pakistan, x1.4 for Brazil and Nigeria and so on (and these are all per capita rates, their total emissions increased far more). Developing countries are catching up fast.

Let's look at past performance of world CO2 emissions. Between 1992-2007, world's emissions increased by 38% - mostly driven by rapidly developing countries like China and India. Even developed countries weren't terribly successful - US and Japan had significant increases in emissions, and EU's stayed pretty much even, and nominally decreased mostly due to early-90s' collapse of heavy industry in former communist member countries. Even if developed countries got their act together and implemented reductions that were agreed on in Kyoto Protocol, they would still be swamped by just Chinese increases, resulting in net emission increases.

And the outlook for the next couple of decades doesn't look terribly promising. Intrade markets, thin as they are (and forgive me lack of links, but their website is virtually unlinkable), think it's extremely unlikely that China and India will agree to any CO2 limits anytime soon, and have pretty low opinion on what reductions developed countries will agree to - chance of agreeing to 10% reduction by 2025 seems to be about even, and there's quite a big difference between agreement and implementation (especially due to possibility of accounting shenanigans of land use changes and such), and such reductions would be swamped by developing countries increase. And in any case the 74.5 billion tons number already assumes that developed countries average will get reduced to current EU levels.

If you hope on peak oil to make 74.5 billion tons/year kind of number impossible, not only peak oil doesn't seem to be happening, coal production grows insanely fast, and estimates of "peak coal" talk about mid 22nd century.

Alternatives to 74.5 billion tons a year

So with business-as-usual, we'll be emitting enough CO2 to get the entire planet stir-fried. What would be alternatives?

First, either outright genocide or indirect genocide by limiting economic growth of developing nations - remember that poverty kills hundreds of millions of innocent people and causes vast suffering. Not only is it unlikely to happen, it would be probably the worst scenario of all.

Second, learn to live in a warmer world. This isn't necessarily going to be as bad. It's hard to imagine anyone who would like to have their children die of malaria and diarrhea just to satisfy some rich smug liberals. People who measure costs of limiting CO2 emissions usually focus on limits in States, or EU - but this is nonsense! Big increases don't come from SUVs, but from bringing poor people to more civilized standards of living, and limiting that would be disastrous. Costs of just taking global warming would be much less than the genocide that would ensue to keep poor people in their place. I think this scenario isn't entirely implausible - there were quite a few cases in history where international cooperation completely collapsed, and in externalities cases like CO2 emissions, this is a possible result.

The third solution, and we're getting somewhere now, would be to sever the link between GDP and CO2 emissions. This is surprisingly hard. Yes, it's trivial to move from Hummer to Prius, but the problem is people who cannot afford either yet, but will want some means of transportation eventually. And they will want electricity - coming from coal most likely. The alternatives look pretty bleak - nuclear power, and large-scale hydroelectricity, both provide only 3% of total primary energy use now (or about 15% of electricity each). All fancy types of renewable energy together don't even add up to 1%. A surprisingly large amount of energy is "traditional biomass", or wood and agricultural waste used for heating and cooking (but then, estimates of it seem to vary wildly). Other than that, right now we're entirely dependent on fossil fuels.

A popular idea of high standards of living without corresponding increase in energy use seems pretty unlikely to me. Yes, we might be able to break the correspondence at some point, but energy use per capita in poor countries is so low, it's simply bound to drastically increase.

Much more promising path is energy generation without fossil fuels. Our best chance so far was with nuclear power. Unfortunately we suddenly stopped what looked like extremely rapid exponential growth in late 1980s after Chernobyl disaster - estimated 4000 people will die due to increased radiation exposure (much higher estimates based on linear no-threshold model can be safely ignored, as LNT model is known to be empirically completely wrong).
By ridiculously naive estimations, if this growth continued exponentially as it seemed to have up to Chernobyl, nuclear energy would provide around 20% of total energy production today, or half of all oil could be eliminated. Now this kind of extrapolation is ridiculously naive, but you should give it some thought.

So could we switch to nuclear at ridiculously rapid pace? We would need over 50 times as much nuclear capacity to generate enough energy to cover increased energy use by 2050, what seems ridiculously optimistic by any standards, not to mention nuclear proliferation issues. It looks somewhat less ridiculous when expressed as 10% a year growth (or doubling every 7 years like pre-Chernobyl), but it's just not going to happen.
There aren't that many renewable options. Current global energy consumption is 15 TW, and it will be about 39TW by 2050 by extrapolation. The only large scale renewable energy source we use, hydroelectricity, is nowhere close to our needs, and neither is geothermal really, if we look at what small fraction of it we can practically extract thanks to high local concentrations. We're left with just two then - wind, and solar. We would need to use 5% of all wind energy of Earth to fill the energy gap.

In 2008 wind power had 121GW of installed peak "capacity", and produced 260 TWh of energy (what corresponds to 30GW of actual production). It would require order of 1000x increase in wind power, or 19% a year growth, to make it fill the energy gap. And the actual growth is actually fast enough at 29% in 2008. But there are clear diminishing returns here - best locations will be taken, and massive subsidies cannot really scale to TW levels.

The other alternative is solar power. Just Earth-based solar is so plentiful that using even 0.02% of solar power now, or 0.045% of it in 2050, we can replace all of world's energy use. And it doesn't stop there, with space-based solar we can get numbers many many orders of magnitude larger. It would sort of need a space elevator, but we might get there by 2200.

Anyway, while solar is undoubtedly our long term future, it doesn't look that bright in the short term. There's only 15GW of solar thermal installed, and mere 0.6GW of solar thermal. And these are capacity numbers. We're talking x10,000 increase to cover the energy gap, or 25% a year increase, or x200,000, 35% per year for solar thermal. Now these numbers are actually what we see now, but as I said for wind, these are all based on massive subsidies, and there's no evidence that they're scalable to TW range at all.

By the way all this is based on assumptions that all energy is basically equal, and all greenhouse gas emissions is energy production. Neither of these are true - electricity is relatively easy to replace with renewables, at least if we ignore their intermittent character, but transportation much less so. And even with 100% solar electricity-based economy, there would still be some significant sources of greenhouse gases like methane from agriculture - so whatever we estimate here is bound to be overly optimistic.

With all this said, thanks to better energy efficiency, and widespread use of renewables, it's quite likely that humanity energy use by 2050 will be much lower than baseline number of 74.5 billion tons a year. On the other hand expecting it to be vastly lower than current emissions is just extremely unlikely.

And here comes the final solution - without genocides, without unexpected revolution in either nuclear power or renewables, we will have much more CO2 in atmosphere in 2050 than now. And due to massive lag it has, it will keep warming Earth for a very long time after we finally get our emissions to decrease. So if we cannot agree to live in warmer world, we have to geo-engineer to survive the next century without massive and irreversible environmental change.

Nature did some real life large scale geoengineering testing for us, and it works without major side effects (you probably haven't even noticed the experiment in 1991). Economic estimations put it in ridiculously cheap range - some schemes are estimated to cost just 375 million dollar a year, or just 17,000 Toyota Priuses (seems like a highly relevant unit of measurement) - for comparison replacing every American car with a Prius would cost 5 trillion USD, and it would still do very little towards limiting global CO2 emissions. And stratospheric sulphur isn't the only solution, just the one we have best data on (by the way it's not about polluting more, only about moving less than 1% of existing pollution to higher layers of atmosphere).

So the alternatives are:
  • Outright genocide
  • Indirect genocide by forced poverty
  • Accepting the warming
  • Increase in nuclear or renewable energy far beyond anything that can be realistically expected in this timeframe
  • Geoengineering
Guess which one is the least bad?

Now I just need to blog on abortion, and I win blogging controversy bingo.

Sunday, November 01, 2009

Why Malthusianism refuses to die

kitten in a basket by pixn8tr from flickr (CC-NC-SA)
It's astonishing how popular Malthusian catastrophism is, in spite of such spectacular evidence to the contrary. As disaster keeps failing to come, its details are changed, but the core prediction that we're all doomed doesn't budge. In this post I'd like to explore some evidence against Malthusian ideology - as with so much evidence to the contrary, it like creationism long ceased to be anything else than ideological view.

Classical Malthusianism

But first, let's say what I'm criticizing. Malthus' idea was that population growth is exponential, while food production growth is arithmetic, therefore we're all going to fall back into extreme poverty due to overpopulation, until we're poor enough to be dying in sufficient numbers to counteract that.

The second part is, and I'm not making this up - "therefore, we should starve and abuse people who lost their jobs in a recession to lower taxes on the rich". That's a forerunner of the Real Business Cycle theory if there ever was one. The predictable result of being tough on the poor was that the working poor were starved to the point of eating pig fertilizer, beaten, and sexually abused.

But that's not the worst of Malthus-inspired politicians - they decided to starve one million poor people in Ireland. During the alleged "famine", Ireland was a major exporter of food, with its food production even at lowest point estimated to be enough to sustain population twice as high as it had. And there was well known solution to such famines already tried in 1782–83 of temporarily stopping food exports, but those good Malthusians would rather have a bit of a genocide than to stand in ways of profits.

Anyway, I'm not here to criticize consequences of Malthus' ideology, just its factual basis. It's been 211 years since Malthus' infamous essay, how far behind are we in food production?

Oh wait, actually food production looks pretty exponential on this graph, while population growth looks rather linear. It's not even that - according to virtually all projections population growth is slower than linear, and world populations are due to stabilize fairly soon, here's Wikipedia:
Unfortunately I haven't found any useful food production statistics going all the way back to 18th century, but by now it should be very clear that these relationships do not hold.

Yet, Malthusianism refuses to die. The most popular mid 20th century's neo-Malthusian Paul Ehrlich famously wrote in 1968:
The battle to feed all of humanity is over. In the 1970s and 1980s hundreds of millions of people will starve to death in spite of any crash programs embarked upon now. At this late date nothing can prevent a substantial increase in the world death rate...
Or more specifically - "I have yet to meet anyone familiar with the situation who thinks India will be self-sufficient in food by 1971" and "India couldn't possibly feed two hundred million more people by 1980."

That was in the middle of yield increases so spectacular they caused major difficulties to even harvest it all (High yields led to a shortage of various utilities — labor to harvest the crops, bullock carts to haul it to the threshing floor, jute bags, trucks, rail cars, and grain storage facilities. Some local governments were forced to close school buildings temporarily to use them for grain storage. as Wikipedia says). India became net exporter of cereals by 1978.
You'd think that after two centuries of predictive failures people would have given up, but no. The latest neo-Malthusian fad just a few years ago was "peak oil". It sort of fizzled since, but in case you forgot, here's an awesome YouTube video describing peakoilnik ideology while dancing:



"There won't be much food anyway, as oil is used to make fertilizers and pesticides. Up to two thirds of the world population may not make it. Four billion people may not survive."

Because as they say, we're basically "eating oil" as the meme says. It doesn't bother them much that oil is used at no step of fertilizer production - it's all natural gas, and entire 3-5% of world natural gas (1-2% of world energy use) is spent on fertilizer production (not even as energy source, more as convenient hydrogen source easier than electrolysis of water), but alternatives based on coal, and hydroelectricity are well known and used depending on local availability. China already produces most of its fertilizer with coal. The first commercial hydroelectricity-based fertilizer plant was opened in 1905, over a century ago, and in case you're wondering use of electric plowing equipment dates to late 19th century.

Pesticide production is so ridiculously low, by some back of an envelope estimates takes about 0.06% of world's oil, and is trivially replaceable by coal if need be (I know of no serious estimates, as it's too small to bother anyone with a clue).

What a Malthusian world would look like

So why so many people believe in Malthusian-inspired catastrophism? I have a two-fold hypothesis. First, some people just enjoy beliefs that we're all going to die. It seems that virtually every disaster scenario, no matter how plausible or not - attracts countless believers.

But there are people smarter than that who still believe in Malthusianism. Here's the second part of my hypothesis. When people look at history, to see if it matches some theory, they just look at things that would be observable if theory was true, and see if there are present. They don't do proper Bayesian updating.

Here's a simple example - let's say you have a theory which says that clash of civilization causes warfare. Then the naive thing to do would be to look at history, note a few wars between civilizations, and take it as evidence for the theory. It would also be wrong, as far more numerous wars were fought within civilizations, and proper Bayesian thinking must look at both.

So let's take a look at evidence for Malthusian worldview against the null hypothesis that food production is just like any other commodity, and not any more connected to population than production of clothes, bricks, or paper.

If Malthusianism was true, food production per capita would be pretty much constant - because any increase in food production would result in increase in population, and decrease in food production would cause mass starvation, and adjust population accordingly. But if Malthusianism was false, food production per capita would also be pretty much constant, because what would be the point of growing more food if there was nobody to eat it? So there's no evidence either way.

If Malthusianism was true, everyone would be poor. Not only is this ridiculously false, now, historical estimates of life expectancies vary wildly from 18 to 60 for the same time period.

Malthusianism predicts almost all food production would be in whatever generates highest number of calories per hectare. Non-Malthusianism predicts food production would be much more varied - and it is so. Just the fact that we're keeping farm animals, which by their nature only destroy and not produce nutrients, proves Malthusianism wrong. There might be exceptions, but most grazing lands would produce more calories when cultivated, not to mention Malthusian ridiculousness of feeding animals human food as we do.

If Malthusianism was true, any sudden decrease in population, due to wars, famines and such, would result in rapid rebound, as agricultural capacity existed to support much larger populations, proven by its previous existence. If Malthusianism was false, population would have no particular reason to rebound. Here the evidence is very clear - there are numerous example of major population loses which were not recovered for centuries. Two best examples affect entire Europe, which used to have high populations at height of Roman empire, then fell to very low levels and didn't recover until 12-13th centuries, almost a millennium later. Second fall was due to Black Death, recovery from which wasn't complete even two centuries later.

Malthusianism predicts that it would be extremely unlikely for vast amount of spare arable land to exist - yet this was exactly the situation for centuries in early Medieval Europe, with vast tracks of unused lands.

Malthusianism predicts that famines would happen during times of higher population a lot, while almost never happening during time of low population. But even a brief look at Wikipedia's List of famines shows plenty of major famines both during population lows of early middle ages (5th century famines in Western Europe, 750's famine in Spain, 809 famine in Frankish Empire, 963-964 in Ireland, 1005 in England, 1016 all Europe, 1030-1032 France, 1066 England again, 1097 France again). There were supposedly 95 famines on British Isles alone during Middle Ages. In spite of very low population levels, we can see plenty of them, and we'd probably know about more if it wasn't for horrible Medieval record keeping. Notice that during all that time there was plenty of undeveloped arable land!

During second period of low populations after Black Death, there are major famines 1390 in England, 1481-1483 in France, 1504 in Spain, 1518 in Venice and so on. I'm not even going to mention that modern periods of European population explosion had fewer famines than any other time in history.

Non-Malthusian view correctly predicts that famines and population levels are completely uncorrelated. Famines are caused not by the mythical "overpopulation", but commonly by wars, government action (like in Ireland, and 1930s' Ukraine, notice how democratic governments have virtually spotless track record of no famines, in spite of plenty of agricultural disasters), and most of all - simply by major delay between increase in agricultural inputs and outputs. It doesn't matter that there's plenty of unused land around, developing it would take years, and plenty of labour, capital, seed, and farm animal input, before any food was grown on it - all of which being in short supply during famine anyway. By the time food was growing the famine would be long over. To give you some indications of time scales involved, settlers in new developments were freed from any taxation for up to 24 years. During that time net effect on food production would be negative.

Most of all, current overwhelming explosion of wealth is completely and utterly incompatible with Malthusianism. Even the second worst country in the world - Angola - has life expectancies far greater than world had 100 years ago (one tiny country which doesn't has 26.1% HIV infection rates, and is too poor to afford HIV drugs - hardly a Malthusian reason). In spite, or maybe even due to, population explonion, world's average life expectancy is 66.57 now. This is level not reached by even the richest countries before 1930, and by most developed countries well into 1940s an 1950s.

However hard I try, I cannot find any evidence for even very broadly constructed Malthusianism that cannot be explained as well or better by treating food as a normal commodity. So worry not - we're not doomed.

Widespread capital punishment never existed

Killer Smile by sbluerock from flickr (CC-NC-SA)There's this fancy Western European idea of abolishing death penalty. Wikipedia even has a nice map, with blue countries having abolished death penalty entirely, red still carrying it out, and green/orange having it on the books, but not in practice.
But reality is different than that. Most of the red countries almost never carry out the death penalty, reserving it for extremely unusual cases - in 2008 estimated 5727 people were executed, over 80% of them in mainland China, other than that only Iran and Saudi Arabia doing three-digit numbers. Other big red countries like India, USA, Indonesia, Pakistan, Japan and so on very rarely execute people.

As 59 million people die every year, your chance of getting executed is like 1:10,000, or more like 1:200,000 if you don't like in relatively execution-happy China, Iran, or Saudi Arabia.

Numbers like this are ridiculously uncommon - about 100 people die every year from peanut allergy in just United States. And "accidental suffocation and strangulation in bed", which sounds like a codeword for "BDSM gone awry" causes 327 deaths annually in just USA.

It's not just due to lack of crime - annually, half million homicides take place, and many countries have or had death penalty for lesser things. So if you're a killer, the chance of getting a lethal injection for it are 80:1 (assuming they get the right person), or more like 1500:1 unless you live in one of the three aforementioned countries.

Add to that the usual overconfidence criminals have, and it's clear that fear of death penalty is the least of criminals' concerns. Especially since even in United States you're over three times as likely to die as a result of a shootout with the police than due to legal execution, so we're talking at most 30% extra deaths factor here. The idea that people will stop carrying out crime due to fear of capital punishment is just highly dubious.

History lesson

So is this it? Did the abolitionists win? Not necessarily. Googling indicates that capital punishment for crime was never particularly widespread. Yes, governments killed loads of people - massacres during wars and occupations, killing political opponents, summary executions to quell social unrest, and some such. But legal capital punishment of citizen committing ordinary crimes after proper judicial process? Very unlikely.

Here's some statistics. Stalin killed about 60 million people, and Hitler about 20 million (estimates vary considerably, but let's take those numbers). How many actually received death penalties? Only 40 thousand from Nazis (1 in 500), and 750 thousands from Soviets (1 in 80). And these were mostly political opponents, war and civil war cases and such - not proper criminals.

How about other countries? Czarist Russia averaged less than 60 a year, and remember that many of them were political cases. Between 1770 and 1830 England had allegedly bloodiest criminal code on Earth, with death punishment for "writing a threatening letter", hanging out with gypsies, or "strong evidence of malice" and over 200 other crimes, while still averaging barely above 110 executions annually. Even the infamous Spanish Inquisition barely managed 20 burnings a year (but then, these were mostly religious and political opponents not criminals, so maybe they shouldn't be included here).

It's just really difficult to find any government, that routinely carried out large number of legal executions of criminals during time of peace.

When governments kill

That's not to say governments don't kill. But these killings rarely affect common criminals. Nazis killed people for being of wrong ethnic group. Soviets killed people for being of wrong social class, or just because. The same 18th century United Kingdom which was so cautious with criminals of right ethnicity, killed million people for being poor and Irish (and no, it wasn't any Malthusian bullshit, Ireland was a major food exporter even during this politically caused famine).

Even countries which don't execute criminals at all like Denmark with no executions since 1892, change their mind during and briefly after war time, when 54 people were executed (it's not really a legal capital punishment, as it's all ex post facto). Likewise in Finland, after not executing anyone since 1825, about 550 people were killed during war time (unlike Denmark where it was mostly proper war criminals and people supporting Nazis, here you could get a firing squad for conscientious objection, and without even a real trial).

What does it mean for capital punishment debate?

Other than perhaps in mainland China, Iran, and Saudi Arabia, the debate is purely speculative now. There is virtually no capital punishment anywhere. And other than during wars, occupations, major civil unrests and alikes, it was extremely rare for many centuries. Before that, executions might or might not have been more frequent, but it's hard to talk about proper legal proceedings, so they don't count.

People who support death penalty have a big problem, as there won't be any statistics showing serious effects of widespread application of it during time of peace, as it never happened. There are some bullshit statistics, like those in the original Freakonomics book (very entertaining book, by the way), but due to vast non-random annual variations in number of homicides and executions, we're talking about effects of a rounding error, and social science just cannot do anything useful with them.

So it's mostly about what makes you feel better. If you're a real Christian, then you should follow Jesus and be against death penalty. If you're a fake Christian like Southern Baptist, you should support death penalty because you hate black people more than you love Jesus. And if you're an atheist, do whatever you want, it's not like I can convince you.

Here, I solved it for you.

Wednesday, October 28, 2009

Medieval 2 Total War Concentrated Vanilla 0.07

Charley, the Shetland Pony, bowling :-) by Deep Frozen Shutterbug from flickr (CC-NC)Here's another release of my Medieval 2 Total War Concentrated Vanilla mod.

Old releases suffered from the too-much-money problem, just like the pre-crash Goldman Sachs, so I decided to scale it down a bit. Instead I introduced massive free garrisons.

Here's the full list of changes relative to vanilla:

  • For more dynamic game, all units move 75% faster on strategic map.
  • One big problem with vanilla was that money didn't matter. No matter if you had a lot or not, there wasn't all that much to spend it on. So now all buildings take 1 turn to build, but are 50% more expensive. This drastically increases importance of money.
  • To add some strategic richness, mines give 2x the profits, and cost 3x as much (x2, +50% like all other buildings). Some settlements with rich mines like Zagreb or Vienna are worth a lot, maybe even worth starting a war over.
  • There used to be 50% increase in settlement and merchant trade, and mine profits used to be 3x, but this was a serious overkill. If you manage well, settlements and merchants will give you more money than in vanilla as you can build relevant buildings faster.
  • There also no longer double population growth penalty/bonus for tax rates - it was an overkill.
  • To help small nations, King's Purse increased 2x (that is - free money a nation gets unrelated to its settlements - from 1500-2500/turn to 3000-5000/turn).
  • To reduce annoyance with rebels, rebel and pirate spawn rates reduced 10x. (I'm not actually sure if it reduced them as much as the file says it would, but it feels more or less right now).
  • Cavalry, especially heavy cavalry was seriously overpowered. Small groups of knights could easily crush infantry armies ten times their budget, so infantry was useless other than free militias to patrol cities, and cheap grunts to operate siege equipment. Because with cavalry-centric warfare missile units are useless, this limited army composition even further - and destroyed rich tactical variety that M2TW could possibly have. So all cavalry is now 50% more expensive to recruit and upkeep. Cavalry is still extremely powerful when used right, except now other kinds of armies are useful too.
  • Bodyguards were particularly overpowered - in early periods they simply destroyed everything, were numerous, self-healing, essentially free (you get new candidates for adoption if you lose old generals all the time), and as every faction has identical bodyguards they severely reduced tactical richness of the game. So bodyguards units are the half size now (upkeep cost is proportional to unit size, and they're cavalry, so mod's bodyguard unit costs 75% of vanilla's). Old versions of the mod reduced their hitpoints from 2 to 1, but it no longer happens, as this made bodyguards overnerfed in late periods. Unfortunately there no way to get the balance right - as bodyguards stay the same, but other units massively improve, either bodyguards will be too strong early on, or too weak late on. I moved the balance point somewhat towards the weakening.
  • Vanilla sieges are ridiculously easy, with a single unit of ballistae being able to take down large stone walls. This made sieges was too easy, and as a consequence turned the game into a siegefest. So now gates and walls (but not towers) are 5x stronger. Serious artillery will be needed to take anything but the weakest walls.
  • Towers now have 2x fire rate for normal ammo, and 1x fire rate for flaming ammo. Increasing flaming ammo rate too much as old versions did made siege equipment like siege towers almost useless. But high normal fire rate should help defenders a lot, even for undermanned settlements that AI leaves. This wouldn't be necessary if AI was competent, but we know well it's not, and leaves valuable settlements defended by one unit of Peasants far too often. This balances AI stupidity considerably, and also frees the player to play more strategically by making defense easier (this is not as essential as in old versions of the mod due to increased free garrisons).
  • Tower activation range is 8x larger (a bit of reversal to Rome Total War). So you cannot shut down towers just by scaring off defenders with some arrows like in vanilla. Strong walls, and massive towers' firepower together mean sieges will be very difficult. Prepare a lot of artillery, or spies, or massively overwhelming force that can take severe loses before giving up. I'm yet to take a well-manned Fortress in this mod (but then AI often mans its settlements insufficiently, and there are other options like waiting their food supplies out, or provoking a sally).
  • For consistency with making sieges harder, spies cost 2x to recruit and upkeep. And AI is quite competent at counter-espionage operations, so spy flood doesn't work all that often.
  • To help defenders in sieges, missile infantry ammo increased 2x. It very rarely affects field battles, as in those they rarely get to expend even half of their ammo.
  • A big change is doubling free garrison in cities. Village 0, Town 4, Large Town 6, Small City 8, Large City 10, Huge City 12.
  • The biggest change is introducing free garrison in castles. Motte&Bailey 1, Wooden Castle 2, Castle 4, Fortress 7, Citadel 12. In vanilla castles are either left barely defended, or are massive bleeding money hole - failure both ways.
  • Just like with cities, castles' basic infantry units can be free garrison, but not cavalry or expensive infantry. Basic infantry is one with morale up to 5, and cost up to 155. So Peasants up to Armored Sergeants, and Peasant Archers up to Genoese Crossbowmen, Venetian Archers, and Hand Gunners can all be free when garrisoned. On the other hand Dismounted Knights of all kinds, and most faction-specific elite units cannot. I guess it might introduce some faction unfairness, but I can live with it.
Overall, what you get is much tactically and strategically richer game, with a lot larger differences between factions, but one that still feels a lot like vanilla Medieval 2 Total War. In the mod defense is easy, so you don't have to mindlessly attack everyone.

The main problem left is the spectacularly stupid AI, about which I cannot really do that much.

I also did some other experiments, that didn't get to the mod:
  • Making ocean-traveling ships available from the beginning (well, as soon as you can build sufficient port infrastructure) enables fights with natives. But Americas are disconnected from the mainland Europe, and it doesn't really feel like it's part of the same game. Try it once to see what fighting Aztecs is like, then forget they're even part of the game.
  • Making Mongols (or maybe even Timurids) come earlier. It's a lot more fun to fight Mongols, but then if they come very early, they will totally crush AI on auto-resolve. Unfortunately auto-resolve favours stronger armies too much, so they won't even suffer too much loses, and you'll have to deal with them yourself. One full stack of Pavise Crossbowmen and Feudal Knights led by semi-decent General (or equivalent) can defeat one full Mongol stack without that much trouble in field, and well defended settlement with ballista towers completely destroys them. It's another thing to try once. I haven't fought Timurids yet.
  • Making Gunpowder available immediately unfortunately counters the effects of very strong walls, and yet good gunpowder infantry and cavalry won't be available until much later. I'm not even sure if gunpowder units are better than plain Pavise Crossbowmen, they don't seem to be from the stats, and I never had an opportunity to play much with them.
Anyway, download the mod here.

Blog posts about previous releases: 0.02, 0.04, 0.06.

Thursday, October 15, 2009

Folk Complexity Theory

flikr1218 by flikr from flickr (CC-BY)There's one kind of posts on this blog that get the most hostile reactions. It's not politics (nobody reads those), flaming programming languages, or computer games - the most controversial subject is Big O analysis! Here are relevant posts:

Basically I claim that Big O is useless in most practical situations. And I get flamed a lot. At first I tried to explain to the flamers how they are mistaken, then I basically ignored them after explanations proven to be pointless, but now I see that it's all just big misunderstanding. What I call Big O analysis is a fairly precise construct of computer science. But that's not how most people use the Big O notation - they follow Folk Complexity Theory, which I want to describe here.

The Official Complexity Theory

First, here's a brief repetition of Algorithms and Data Structures 201, or whatever your course explaining this was.

An algorithm is O(f(n)) if in a particular model of computations, if there exist some constants n0, and c, such that for all input lengths greater than n, computation time is less than c × f(n). A problem is O(f(n)) if in particular model of computations there exists a O(f(n)) algorithm that solves it.

Models of computations are things like different varieties of Random access machine, and different models usually only differ by some logarithmic factors. Sometimes you want to assume that adding two numbers takes constant time, sometimes that it takes time proportional to logarithm of their sum, and details like that. If you disregard logarithmic factors, you don't have to be all that precise about which variety of RAM you're using. That's about it.

In particular, if the problem is only defined for constant-sized input, then by definition it has complexity O(1). There exists some c, such that for all possible sizes, running time of the algorithm is less that c × 1.

So breaking AES-128 is O(1). As is breaking RSA-4096. Breaking RSA-n for arbitrary n isn't, but that's entirely unrelated problem. Breaking AES-n is meaningless concept, as AES isn't defined for arbitrary n.

In particular classes like P, and NP, don't depend much on what reasonable computation model you're using. They're defined on Deterministic Turing Machines for reference (which is polynomial factor slower than RAM), but if something is P on DTM, it's P on RAM and vice versa.

How cryptography handles it

As I got a lot of flames about my claim that breaking RSA-4096 is O(1), let me explain how cryptography handles it. So cryptography doesn't use Big O notation much.

Security is measured in bits - a cryptosystem has k bits of security if the attacker must perform 2k times as many operations as legitimate user to defeat security of the cipher. For some types of security key size and security are related - for block ciphers security = key size, for hash collisions security = key size/2. For public key cryptography these bit measures can be only approximately specified in terms of key size, and whatever formula we have we're completely uninterested in its asymptotic form, as all feasible key sizes are small, and constant factors are of great interest to us.

The Folk Complexity Theory

And now the main subject of the post - The Folk Complexity Theory. When people say some algorithm is O(f(n)), they actually mean that for "normal" sizes of inputs, on actual computers, it's expected running time is between two "small" constants times f(n) times reasonable number of log(n) factors as long as they don't push the constant too high.
  • O is used sort of like Θ of the official theory. Folk O(n) algorithm is not considered to be O(n2).
  • So bubble sort is O(n2), as number of operation tends to be something like 10n2. Logarithmic factor for addressing and arithmetic is not considered at all.
  • Merge sort is O(n×log(n)), as number of operation tends to be something like 10n×log(n). Notice how log factor for number of operations is included here as it's significantly higher than allowed small constants; but log factor for cost of addressing and arithmetic is not, as on normal computers it's constant up to 232, which is very big n. And only doubles for 264, which is too ridiculously high n to bother. Exactly the same reasoning can be used for both log factors, so this is highly peculiar.
  • Quick sort is O(n×log(n)), the Folk Complexity Theory always deals with typical case, not worst case.
  • Breaking AES-128 is O(2n) because the constant is too big to be allowed. It doesn't matter that there's no n here, and AES is not defined for other key sizes. The problem must be reformulated to have n, no matter if it makes any sense or not. Number of operation like 2140 is simply "not allowed", while 212×2n is.
  • Breaking RSA-4096 is likewise O(2n), even though there's no n here at all.
  • Forging signature of n-byte message signed with RSA-4096 is, uhm... O(2n). Wait, n is already something else... O(2k) then maybe? The problem is reformulated as RSA-k because large constants are not allowed, and usually there's no point saying O(2k+n), as n is too small to matter for all practical n.
  • As an added bonus "NP" usually means Θ(2n), for real or imagined n. This is spectacularly incorrect (most problems are not decision problems; there can be no Θ bounds even for NP-complete problems until NP != P is proven, and every P problem is NP), but doesn't cause much confusion in practice.
This is the source of all confusion - I was talking about the Official Complexity Theory, while most Redditors interpreted it as Folk Complexity Theory statements, in which they were obviously untrue.

Is Folk Complexity Theory useful?

It's quite amusing that so many people flame me for saying the Official Complexity Theory is overrated, and don't even know it. But I'm not going to blame them - who needs it once they're past exams anyway? In practice Folk Complexity Theory is actually better heuristic than the Official kind. We program real computers. On real input sizes. ceil(log2(n)) factors shouldn't be ignored, but ceil(log4294967296(n)) factors which represent cost of arithmetic and addressing on most 32-bit computers hardware totally can and should. And so should all log(log(n)), which nobody ever bother with past exams.

The Folk Complexity Theory has weird status - it's neither mathematics like the Official theory, nor practice like actual benchmarking. It's just a bunch of heuristics dressed as something much more formal than they are. Often even fairly useful heuristics. Just nothing more than that, and not worth the flames.

The Web 2.0 of Metaphors

So you think you can dance by fofurasfelinas from flickr (CC-NC-ND)After hearing that something is Wikipedia of whatever it is about, and getting slightly annoyed by it, I decided, in interest of science, to compile popularity ranking of "the X of Y" metaphors, by Google counting.

I found quite a few false positives, like "the livejournal of Nikki", and "the Linux of your choice", but upholding the glorious scientific tradition of ignoring every methodological issue that cannot be fixed, I completely disregarded them.

  • "the windows of" - 15,100,000
  • "the internet of" - 7,790,000
  • "the apple of" - 6,470,000
  • "the google of" - 3,160,000
  • "the youtube of" - 766,000
  • "the microsoft of" - 397,000
  • "the imdb of" - 307,000
  • "the craigslist of" - 303,000
  • "the livejournal of" - 290,000
  • "the ebay of" - 277,000
  • "the amazon of" - 262,000
  • "the facebook of" - 237,000
  • "the myspace of" - 235,000
  • "the linux of" - 197,000
  • "the wikipedia of" - 180,000
  • "the yahoo of" - 142,000
  • "the flickr of" - 124,000
  • "the ibm of" - 109,000
  • "the blogspot of" - 108,000
  • "the web 2.0 of" - 92,900
  • "the bing of" - 90,900
  • "the firefox of" - 63,200
  • "the aol of" - 63,100
  • "the napster of" - 61,200
  • "the msn of" - 52,100
  • "the digg of" - 23,600

It seems the most popular activity is explaining one web service as metaphor of another:

  • Is Craigslist the "Napster" of the Sex Industry
  • YouTube Redesign: Becoming the Google of Video?
  • Is Facebook: the Google of Social Networks
  • Google Earth Is the AOL of the Geoweb
  • MySpace is the AOL of Social Media/Web 2.0
  • YouTube.com — the Flickr of Video

There are also some unusual finds:

  • The Wikipedia of Slutty Dressing
  • The Craigslist of Antibiotic Resistance
  • Is Python the Apple of Programming Languages?

Another interesting find is that while I've been expecting a lot of "the X of sex", only Wikipedia (36,200) and Facebook (29,500) have decent number of matches.

"The X of porn" is a lot more popular, here Google (378,000), YouTube (354,000), and surprisingly Napster (22,200) are taking the lead.

The other most popular Internet activity, spam, got almost no hits.

Wednesday, October 14, 2009

Efficient go hypothesis

Keep walking by fofurasfelinas from flickr (CC-NC-ND)Efficient market hypothesis states that you cannot really beat the market by much, all the prices are about right, and everybody behaves it more or less rational way. Yes, treating it too seriously leads to ketchup economics thinking, but it's often a good starting point - if you postulate that some prices are seriously wrong, you better come up with a story why they're wrong.

But let's forget about economics for a moment. Something very similar to efficient-market hypothesis applies to the game of go. Unlike most other games like chess and football, go is extremely balanced.

  • Black player, who moves first, pays 6.5 points for the privilege, what makes both players' chances of winning as close to 50:50 as it gets.
  • If one player is stronger, the weaker player gets as many free stones as necessary to get the chance of winning very close to 50:50.
What's interesting is that during the game, it's impossible to make a move that highly increases your advantage, or "beats the market" - if you could get significantly ahead by playing a stone, then the position was already unbalanced before you've done so due to this possibility. Values of positions consist of complex balance of territory, influence, flexibility, good and bad aji, and many other things - if you ask a computer program like gnugo to estimate balance of the board, you will get widely varying numbers with each moves, each totally ridiculous.

Just like on the market we cannot give correct formula for a price, but we know it's right because people trade at it; in go we don't have a formula for value of a position, but we know it's even if good moves by two players led there. If you postulate imbalance, you better come up with a story telling which move was bad and why.

Go players often perform some complex trades of moves called joseki - standard sequences that lead to more or less even results. Again, why even? Because neither you nor your opponent would purposefully play bad moves to fall behind, and if there are no good moves available it means one of you was already losing earlier.
Joseki example - every single move here and the sequence are about even
This requires significant shift in thinking compared to games where you act to get as far ahead of your opponent. You can play go like an index fund - just try to stay even with each move, and let your opponent make some mistakes and fall behind you.

That doesn't mean there's no room for creativity - sometimes there's only one good move to make, most of the time you have multiple good options with subtly different balances. These tiny variations are enough to win - there is no possibility of draw in go (not counting super-ko and such exceptional situations), so even if you get just 1 point ahead through the entire game, 0.01 point advantage per move, that's just enough. It's kind of like high frequency trades, which earn tiny profits, but with sufficient volume it's worth it.

Most professional games end with just a few points advantage for one side or the other. Or one of the players falls behind just a bit, and then risks it all on a great all-board fight to avoid the few point loss.

So as you can see, go is just like market. Except it never leads to crises costing trillions of dollars.

Sunday, October 04, 2009

Practical O(n) problem impossible to solve even for n=1

Pinky by swanky from flickr (CC-NC-SA)
In computational complexity a fairly popular claim seen in most textbooks equates class P with practically feasible algorithms, and everything not known to be P (but usually within NP, "not known to be P" is an awkward thing we need to say until P!=NP is finally proven, even though we all know it to be true already) with not practically feasible algorithms. While it's often noted that it might be untrue for some cases, with some P algorithms being impractical, and some not known to be P algorithms to be practical, such disclaimers usually consider these cases to be contrived and not worth any further consideration.

Here's a really simple example of a genuinely useful problem in P (linear even) that's not practical at all.

Utterly impractical O(n) problem


Imagine problem of falsifying digital signature for a message in some reasonable digital signature scheme. Inputs are public key of constant length, and message of length n. The algorithm is as follows:
  • Extract private key from the public key by brute force, or some other algorithm (constant time)
  • Sign the message (linear time in n required just to read the message, and no more than linear time necessary)
So we have an O(n) algorithm, and because we need to read entire input to sign the message, no faster algorithm is possible. Therefore the problem is Θ(n).

Yet, unless someone breaks the public key system and provides vastly faster method of extracting private key based on the public key, we won't be able to solve it even for n=1.

This problem is of great practical importance - the entire online banking system would fail, and billions of dollars of fraud would happen if someone could practically solve it. So it's not fair to call utterly unfeasible P problems contrieved.

Thursday, September 24, 2009

Google Everything and some alternatives

Search cat by zenera from flickr (CC-SA)
Do you feel a bit uncomfortable using countless web services all operated by a single company? I can feel your pain. Here's a list of things I use (or tried but didn't like), and some alternatives if I know any.

Google Search

It used to be the case that Google Search was vastly better than anything else. Not longer so.
Don't believe me? Try the Blind Search - it searches the same term in Google, Yahoo, and Bing, then you can vote for the winner. Most of the time the results are extremely similar, and the scores are Google - 41%, Bing - 31%, Yahoo - 28%. Google still leads but not overwhelmingly.

While with text searches there's no reason to switch away from Google, Bing Images and especially Bing Video is quite significantly better than Google equivalents. Bing is simply a better search engine for porn than Google, they know it, and even advertise this fact:


I recommend we start using the word "to bing" as "to search Internet for video porn", just like "to google" means "to search Internet for textual websites".

Google Mail

I tried plenty of other webmail providers, and plenty of offline email programs. Even some Intranet email solutions like Zimbra and Exchange. Google Mail is so overwhelmingly better than anything else it's not even funny.

Hotmail, and Yahoo Mail, which I tried as throwaway spam accounts, are just too painful to use. There are no alternatives, it's Google or pain.

Google Maps

Google Maps was quite revolutionary when it first appeared. Unlike with Google Mail, here the competition did catch up quite successfully. Yahoo Maps, Bing Maps, MultiMap, StreetMap (well, that one's UK-only), and probably some others, are all quite usable for directions, and all suck for searching local businesses.

I've heard some drivers having preferences for one or the other, but as a public transport user/occasional cyclist, I'm still waiting for the first to offer decent TFL integration.

YouTube

YouTube annoys me considerably - they don't want to host anything even remotely risky. They limit access to videos by geography, remove those that include some copyrighted music in the background to unrelated funny thing.

On the other hand they have no problem hosting this video of a brutal rape (NSFW).

There are some alternatives, none anywhere near the size of YouTube. Not counting the porn tube of course. I have limited experience with them, so no in-depth comparison here.

Google Chrome

It's nowhere near being the most popular browser. I'm also not sure what's its main selling point, other than tab sandboxing, which Firefox badly needs.

Other than that, Firefox is vastly better. It has countless useful plugins. It has Greasemonkey, and Greasefire. It has Firebug. It has AdBlockPlus. Chrome isn't even close.

Not like I'm complaining, if it can get even one user away from IE, it's a win.

Blogger

Let's face it - Blogger is a pretty crappy blogging platform. But then I haven't heard users of any other being happy about theirs, so perhaps all are equally bad. And effort of moving the blog somewhere else (plus losing the URL) seems too big for me to bother.

In particular the way they handle spam comments is simply atrociously bad. I get a notification of a comment, and then it's about ten clicks to remove it if it's spam, instead of just one or two.

Google Reader

It's basically an online RSS reader. What I hate about most desktop RSS readers is that they tend to push notifications onto you, and don't live inside the most natural place possible - the browser.

With Google Reader Watcher, Firefox tells me how many unread items there are, and which website they're for. Which is exactly as much notification as I need.

I think the only thing left to want is "open every new item in a fresh tab" function, but there's probably some Greasemonkey script for that, or if not it would be pretty simple to write it.

I don't see much need for alternatives.

Google Talk

The main problem with IM programs is that everyone is on a different network, so you need way too many programs to talk with everyone. Adium has most of them. Except Skype. And the most basic functionality only - not webcams, no microphones, not even all the icons.

I don't have particular problems with Google Talk, but I'm going to use this unique opportunity to rant about Skype a bit. OSX version of Skype doesn't seem to have any way of disabling text notifications of contacts becoming available or unavailable. So it constantly spams me with irrelevant garbage. It's not possible to use Skype network from other programs even for IM chats. Many networks try this restriction - and abuse their control over access to other users, but Skype seems to be the only one which did it successfully. Would you be happy if you needed an iPhone to call a person using iPhone, a Nokia phone to call a person using a Nokia phone, and basically 20 different phones to speak with everyone you want to? Me neither, so why are we allowing exactly the same thing in the IM world?

Google News

I don't see Google News as solving any real problem. My main problem with news is that they focus on events of the day, while the important things are usually long ongoing processes, with little need for daily updates. And if you're away for a week or two, there's no easy way to look back at this time and see if anything important happened.

As far as I can tell, it's equally true for all alternatives. If you know a good way to follow important news, tell me about it in comments.

Google Docs

Google Docs is a pretty cool webapp for editing documents together. I have irrational hatred towards WYSIWYGs, especially web WYSIWYGs, so don't ask me about fancy advanced features, but it solves the basic need well enough, and I don't see what could be the big selling point for its alternatives.

Google Code

Google Code is a nice hosting service for Open Source projects. It supports Subversion (obviously), and Mercurial, but not git.

Some of the alternatives are SourceForge (both Subversion, and git, also some other VCSs I don't care about), github (only git, duh). I'm not sure how bad github's limitation of 300MB per open source repository is - my personal code repository's .git is easily bigger than that, but then I freely store binaries and stuff in it (like font files for jrpg).

Overall I don't really see any major advantage for Google Code over it's countless competitors, and it's not even particularly big.

Google Analytics

I'm going to be pretty clueless about it, as Google Analytics is the only one I tried.

Good things - it gives you pretty decent overview of traffic, and is really easy to install.

Bad things - no decent API (screen scrapping possible but quite painful), no access to raw data, I'm not a big fan of their interface, which seems to be focused on all the wrong things.

If I started a website these days, I would definitely try something else, if only to see if it's any better. Switching an existing site would probably not be worth the effort, as all historical data would get lost.

Summary

These are just some of Google services and programs I tried. It's scary in two ways. First, a single company has way too much control over the Internet, and over people's often private data. Google has been relatively well-behaved as far as huge soulless corporations go.

Second, and perhaps more important problem is - what the fuck is the entire competition doing? The only reason people use Google Absolutely Everything is because most of everything else sucks so much. Google didn't succeed just once or twice, it had long stream of spectacular successes where everyone else failed (search, maps), and often even failed to catch up years later (webmail, text ads). Seriously, how difficult is it to innovate a bit online? Difficult enough that a single company does most of it? Hardly anything good on the web ever came from Yahoo, Microsoft, Apple, Sun, and countless other huge Internet companies in ages. The only other major company that seems to be doing anything interesting is Amazon. A damn bookstore. A damn bookstore is doing more web innovation than all the software and web giants put together.

Yes, there are still startups doing the cool stuff, but most of them stop as soon as they get bought. Like del.icio.us, which was revolutionary, but then got bought by Yahoo, and is stagnant ever since.

In search for Just Wars

Allied World War I soldiers by Dunechaser from flickr (CC-NC-SA)
I know people don't really read my political posts. If I wanted to be read, I'd be writing angry rants about video games, programming, and Harry Potter fan-fiction.

Here's a quick post about politics, namely about just wars, and other violence. Right wing nuts are generally pro-war, no matter who's fighting whom. What's surprising is that even left wing nuts are pro-war. Here's some Chomsky, it doesn't get much left-wing-nuttier than that:

ROBERT B. SILVERS: ... Under what conditions, if any, can violent action be said to be "legitimate"? ...

NOAM CHOMSKY: My general feeling is that this kind of question can't be answered in a meaningful way when it's abstracted from the context of particular historical concrete circumstances. Any rational person would agree that violence is not legitimate unless the consequences of such action are to eliminate a still greater evil. Now there are people of course who go much further and say that one must oppose violence in general, quite apart from any possible consequences. I think that such a person is asserting one of two things. Either he's saying that the resort to violence is illegitimate even if the consequences are to eliminate a greater evil; or he's saying that under no conceivable circumstances will the consequences ever be such as to eliminate a greater evil. The second of these is a factual assumption and it's almost certainly false. One can easily imagine and find circumstances in which violence does eliminate a greater evil. As to the first, it's a kind of irreducible moral judgment that one should not resort to violence even if it would eliminate a greater evil. And these judgments are very hard to argue. I can only say that to me it seems like an immoral judgment.


Oh, really? So, could you please point me to some examples of wars that successfully eliminated greater evils? The criteria are, you can agree that they are sane:
  1. Without the war, very large number of people would have died or otherwise suffered.
  2. There was no other plausible way of saving them, and all alternatives have been tried.
  3. Thanks to the war, most of those people were saved.
  4. Few people died or otherwise suffered in the war, especially civilians, but also participants. Some suffering is allowed, but only if it's vastly overwhelmed by suffering prevented.
  5. It must have been possible to predict it advance that this success was very likely. Lucky wins don't count. And of course, it must have worked. Failed good intentions don't count.
I have really difficult time finding out wars that fit these criteria, even with very liberal grading.

The latest war in Iraq

The latest war in Iraq is easily not anywhere close to a just war. Saddam wasn't a nice guy, but amount of suffering he was causing recently was quite insignificant as for third world dictator standards, so it fails on count one, and automatically two and three.

According to Lancet estimates there were 1.3 million excess deaths due to it. Many other Iraqis suffered in other ways. So fail on count four.

Only fools would have expected an easy win, and being greeted with flowers, it's the Middle East after all. All claims about WMD, Saddam's connections to Al-Qaeda and so on don't count, as they were known lies, and even if Bush believed them, they would fail by count five.

Score: 0/5

The latest war in Afghanistan

This war is slightly better, as number of casualties on all sides is closer to 60 thousand, not counting another 10 thousands or so in the spill-off to Pakistan, what's most likely an underestimate in a poorly researched place like Afghanistan, but still much lower than Iraq's. And Bin Laden was in Afghanistan.

Still, it fails on all counts. There were people who died in Al-Qaeda terrorism, but other than a single well-known case, repetition of which was fairly easy to prevent, their numbers were really low. Even with 9/11 much lower than number of people who could be expected to die in the war, so count one fails.

There were plenty of plausible alternatives for getting or killing Bin Laden, and dealing with terrorism, so fail at criterion two. See IRA and ETA for good examples. Or even Clinton-style targeted killings.

Terrorism increased greatly after the war, so it fails criterion three. Count four doesn't fail that badly, assuming figures are not overly underestimated (they usually are, counting only direct deaths), but it's still pretty large number just to catch one guy. Let's be generous and count it as a point.

Anyone who expected an easy win in Afghanistan must have slept through history classes. And Afghanistan is still in chaos, terrorists still terrorize, and Bin Laden was never caught or killed, so criterion five spectacularly fails.

Score: 1/5

The Second World War

Let's move back in time to World War II, widely proclaimed to be the most just war in history, without any basis in fact.

So first, suffering. This is very likely - if everyone just gave Hitler what he wanted (mostly Central Europe to share with Stalin), millions of people would be reduced to status of second class citizens, and depending on how it would go, hundreds of thousands or even millions would die. It's not certain if large scale death camps would be implemented without the war - it seems Nazis were fine with Jews and everyone else they didn't like just getting out of their country (500,000 or so emigrated from Germany before the war, also see the famous Madagascar Plan), but let's assume they really wanted to perform massive genocide. In any case, the first criterion is quite plausible, so full point.

The second criterion fails. The obvious plausible alternative of letting people persecuted by Nazis emigrate to other countries, most were not terribly happy about letting them in, even through Nazis were quite glad about just kicking them out (first confiscating most of their property). Forced resettlement would suck, but it still beats getting gassed.

Now, the third one fails miserably - Nazis killed pretty much everyone they wanted to kill. Most people who would have suffered living under Nazi dictatorship, were instead forced to live under Soviet dictatorship. Approximately 70 million people have died, most of them outside territories that Nazis would have if the world decided against going to war with them. So it's spectacular failure on the counts three and four.

There could be no expectation of an easy victory - spectacular fall of France might have been unexpected, but World War I style trench warfare with millions of casualties was the primary alternative. It's not even obvious if Allies really cared about defeating Nazis - if they were serious about winning, going after them in September 1939 while they were busy in Poland seemed like the best moment, but they didn't do anything until they were attacked.

Score: 1/5

American Civil War

Here's another "glorious" war - American Civil War. It's really historically inaccurate, but let's grant the common myth that the cause of war was abolition of slavery. There were four million slaves in USA in 1860, to be freed. So a somewhat mythical point, but granted.

As for alternatives being untried, most of the world abolished already or soon after 1860, without getting into any major wars. Even countries highly dependent on slave labour, which quite smoothly transitioned into poor waged labour, with or without compensations for former slave owners. There's no reason why it wouldn't work in United States, sooner or later. No points here.

As for point three, yes, all slaves were freed, so entire point here. Sure, there was a century of discrimination ahead, but let's not get too nitpicking.

Point four utterly fails. 620 thousand soldiers were killed, plus some unspecified number of civilians. It was quite clean as far as wars go, so I wouldn't expect a huge value for civilian excess deaths. Still, that's only 6 slaves freed for one death. I'm sure most abolitionist activists had far better than this number - by collecting money, and buying off slaves' freedom, or by nagging slave-owners' conscience until they agreed to free slaves one person could surely free far more than 6 slaves with lifetime's effort.

As for point five, it was successful, and due to North massively overwhelming South in industrial capacity, military strength, and population, it could be expected to be an easy win. It wasn't all that easy, but point granted.

Score: 3/5

American War of Independence

American Revolutionary War is another big one, but I really don't see why.

First, suffering - there were some extra taxes white protestant elites weren't terribly fond of. That's zero points, or else Washington, D.C. has as much right to rebel right now.

Second, alternatives. It's automatic fail due to lack of significant suffering, but other countries like Canada successfully got as much autonomy as they wanted without armed rebellion.

Third, prevention of suffering. Also automatic fail, but notice how United States kept slavery long after the rest of British Empire abolished it. It's not at all obvious that it was more representative, or more democratic than other colonies.

Four, casualties. There were about 52 thousand soldiers killed, plus unknown number of civilians (but not too many, as it was one of the rare clean wars). Vast majority of those were from disease, most commonly scurvy, a disease that was already known to be easily preventable. Battle deaths were barely over 10 thousand. That's small by modern standards, but not trivial relative to two million people living in rebelling States back then - just about 2%. I'll grant a point, but it's just being generous.

As for point five, expecting an easy victory against British Empire at peak of its power was foolish. It was a lucky win. No points.

Score: 1/5

Cambodian–Vietnamese War

That covers the usual suspects, which I all mentioned mostly to point out how they fail at being just wars. Now let's look at some obscure wars that might actually succeed.

Cambodian–Vietnamese War is my best candidate. Khmer Rouge killed approximately 1.5 million people out of population of 7.1 million, and weren't stopping. They didn't even limit themselves to their borders, but also raided some towns in Vietnam and massacred civilians there. Overall, spectacular victory at point one.

It's hard to tell what alternatives could be tried, but none are obvious, and due to massive urgent suffering we cannot blame the Vietnamese for not looking too hard. A second point.

Pol Pot was successfully overthrown. Guerrilla lasted for another decade, but it was on much smaller scale. Full point.

Point four is not entirely successful, as China attacked Vietnam to stop their Cambodian operation, with wide variety of estimated, but something of order of 100,000 soldiers and civilians killed. Still, that's far less than number of Khmer Rouge casualties, so point granted.

War could be expected to be easy - huge, well equipped, and experienced Vietnamese army could and did easily overwhelm Khmer army, and it did so successfully. Full point.

Score: 5/5

Most other wars of independence

Cambodian-Vietnamese War proves a just war is possible, even if really rare. Let's go back to other wars for a moment. One very popular kind are all independence/liberation wars, and uprisings against dictatorships.

They usually fail on suffering front - most colonial powers don't engage in massive killing, so the suffering is mostly widespread discrimination, and lack of representative democracy.

They fail on the second front, as countless countries succeeded in getting significant autonomy, or independence, by political action. It worked even against the Soviets, so it doesn't depend on the colonial power being "nice".

As for the third point, most governments that get into power after such wars are dominated by military, or otherwise very far from democratic ideal, and hardly much better than previous colonial powers. You might be lucky sometimes, but I wouldn't count on it.

Fourth - there's usually plenty of civilian casualties. Independence wars, unless the colonial power simply decides to give up early, tend to be really dirty. No points.

Fifth - by balance of power, it's really naive to expect a victory here. Yes, sometimes it happens, not that rarely even, but lucky wins don't count. And there's even lower chance of the new government being much better than the one being replaced. No points.

Related reading: From Dictatorship To Democracy.

Score: usually 0/5, occasionally higher score.

Terrorism

If we go even further from guerrilla uprisings to terrorism, it gets even worse.

Now occasionally like in Palestine the suffering is real, so point granted. In many places like Basque Country it's fairly insignificant.

As for alternatives, political methods not involving violence have been quite successful in countless cases. So zero points.

One interesting thing about terrorism, is that is pretty much always fails. Of very long list of organizations employing terrorism, the only totally successful case is Hezbollah, which kicked Israelis out of Lebanon. Tamil Tigers - who had semi-independent state in North Sri Lanka for quite a long time before being defeated recently, and Hamas and other Palestinian groups, which fought out some very limited autonomy for Palestinians, can be counted as partially successful. Everything else was a spectacular failure. Notice how the successful cases are really guerrillas employing some limited terrorism against mostly military targets as a small side activity, so they shouldn't even count as primarily terrorist organizations. Zero points here.

As for random unrelated casualties, they're present by design. Usually they're not terribly numerous compared to a proper war, so actually we could grant half a point here, but that would be stretching it. And that would only count if it worked, what not a single case of terrorism so far did.

As for chances of success - well, so far they're 0 successes in far too many tries, so no points.

Score: 0/5

Bombing of Bosnia and Herzegovina

Let's look at operations like bombing of Bosnia and Herzegovina.

First, suffering. Over 100,000 people died in the war, about half of them civilians, so it was a pretty good cause. Plus millions displaced, usual looting, rape, and what you usually see in dirty wars. Full point.

Second, alternatives. Some effort was taken, and out of three fighting parties, UN managed to get two to agree to stop. There were also attempts to limit civilian casualties by creating safe zones (openly abused for military advantage, but that's a completely different issues). You could argue about lack of fairness by UN, and if really everything that was possible was tried, but I'll grant the full point. Also military action was quite limited, aimed at getting quick political solution, not full annihilation of Serbian military capacity.

Third, prevention of further suffering - that's an easy point, operation was fully successful.

Fourth, no extra casualties - bombing is estimated to kill over 100 Serbian soldiers, a trivial number, and many more of just Serbian soldiers would have died if the fighting continued. Full point.

As for estimating chances of victory... well, wars are rarely won by just dropping a few bombs, but NATO was overwhelmingly stronger than any of the fighting parties in Bosnia, and it was successful, so full point.

Score: 5/5

Kosovo and other operations

While bombing of Bosnia was really successful, other peace enforcement campaigns like one in Kosovo much less so.

There were around 2000 casualties of the war between Serbian military and police and KLA, but no massive genocide ongoing. The best example of war crimes found was Račak massacre of about 40-45 people, and even that isn't really terribly solid. There was killing due to fighting, and large scale displacement, but that's rather underwhelming, so just half point this time.

Alternatives were under-explored. There was some attempt at negotiating, but it's not really obvious if NATO seriously wanted a solution that would be acceptable to both parties, as opposed to just getting some cover. No pressure was put on KLA, even though they would be fairly easy to press. And during the negotiations all sides issued a joint statement agreeing to "substantial autonomy for Kosovo", and "invited international civilian and military presence in Kosovo", so it's not really obvious what the subsequent fighting was about. No points.

Prevention of suffering was also quite unsuccessful, the war erupted even more strongly after bombing started, with some proper massacres this time. 80% of people killed, and 90% of people displaced happened after the bombing started. 250 thousand Serbian refugees never returned to Kosovo. No points.

Bombing as such didn't kill terribly many people, around 500 civilians, but due to massive increase in fighting on the ground huge number of people died. There was also a later spill-over of fighting into Macedonia, but it wasn't that severe. No points again.

I'm undecided if the last point should be granted or not - previous successes like Bosnia suggested that it would be an easy victory, but it worked only so-so, and large scale ground fighting caused a lot more suffering that was prevented. Half point.

Score: 1/5

Summary

According to criteria of successfully and reliably preventing significant suffering, the only good wars I could find where overwhelming force is used to stop already happening war or other large scale atrocities, and they limit themselves to minimalist goals instead of trying major political restructuring.

All wars where one side is not overwhelming, like WW2, and American Civil War, tend to cause massive suffering, comparable or worse than whatever can be prevented, even if one of the sides is clearly doing a lot of evil.

"Wars to End All Wars" are as old as known history, and include not only WW1, and WW2, but events as ancient as civil wars of late Roman Empire, and probably many before it. Success rate - 0%. Using this argument to support any war is just ridiculous.

All fighting from position of weakness - anywhere from terrorism to full scare insurgency - tend to cause massive suffering, and have even few successes at preventing major suffering.

If you know of any other wars that score very high according to my criteria, or have problems with my rating system (ratings are quite subjective, but it's hard to come up with anything much better), post it in comments.

Chomsky is a total idiot

Which all means that Chomsky is a total idiot, as he is a big fan of the kind of unsuccessful wars from position of weakness like Tamils in Sri Lanka (80 thousand or more people killed with nothing to show for it), and a big critic of successful wars like Bosnia (apparently Srebrenica was "not really a massacre" with only 8,000 or so people killed).

Yes, American government engages in many stupid wars, and causes a lot of suffering. But that doesn't mean that everyone who happens to criticize American government is automatically right and entitled to the high moral ground. Reversed Stupidity Is Not Intelligence, as my favourite philosopher Eliezer Yudkowsky says.