Personal experience points and OSX menulets

Doesn't it seem odd how people are willing to spend so much time and effort on doing everything that's best for their in-game characters, and yet they never do anything for their real lives?

And it's not just highly complex MMORPGs like World of Warcraft and Eve Online - look how many people spend how much effort on FarmVille - which is about logging it at scheduled times and clicking harvest/plow/plant seeds on a large number of virtual squares! Many of them are the same people who avoid putting any effort into their real life characters as much as possible.

There's been some interesting discussion about this real world - gaming world divide, started by this TED talk by Jane McGonigal - like vast majority of TED talks it's totally wrong, and totally worth watching for entertainment value and some intellectual stimulation:







By the way - to people who are new to this blog - links here are like on Wikipedia - they usually lead somewhere interesting.

Anyway. Assuming you want to achieve some real life outcomes but your laziness and disorganization stops you - like most people - why couldn't you simply approach life just as if it was a game? Well for one thing games give you a lot of immediate feedback that you're doing well, but real life doesn't.

So why not fix the problem and create such immediate feedback? This "feedback" is usually little more than just some numbers, progress bars, and make some badges or other icons. Let's do it!

Experience points log

Well, first thing you need is log actions which give your real life character experience points.

A log can be a simple semi-structured text file like this.

= RULES =
# Every time
+10 write a blog post
+2 read a book
+1 go to gym
+1 play with cat

# Just once
+100 setup a backup system
+500 organize free elections in Belarus

= 2010-03-27 Sat =
+10 blog post
+1 played with the cat

= 2010-03-28 Sun =
+1 played with the cat
+2 read a book

Example is completely made-up but you can think of some points for yourself. And of course you can go back and change these rules and numbers any time you want.

Recording actions

Unfortunately there are no iPhone apps for that yet, so we'll have to record our actions manually - but let's make it as easy as possible. The log is located at /home/taw/all/xp/xp.txt and I have the bound one of the keys on the most awesome keyboard in the world to open this log in case I want to edit it.

And because I almost always have terminal open, I wrote a script which appends current date header if it changed, and if you passed it any arguments it appends them to the log, otherwise it opens it. Actually now that I think about it, it wouldn't be a bad idea to get the keyboard shortcut to do this date change too... The script also edits itself if I pass it --edit

#!/usr/bin/env ruby1.9
exec "mate", __FILE__ if ARGV[0] == '--edit'

require "time"
require "date"

fn = "/home/taw/all/xp/xp.txt"
last_day = Date.parse(File.read(fn).scan(/^= (\d{4}-\d{2}-\d{2} \S{3}) =$/)[-1][0]) rescue nil

out = []
unless last_day == Date.today
  out << Date.today.strftime("\n= %Y-%m-%d %a =\n")
end
out << ARGV.join(" ") unless ARGV.empty?
unless out.empty?
  open(fn, "a"){|fh| fh.puts out.join}
end

system "mate", fn if ARGV.empty?

It's all highly portable so far as long as you adjust paths etc. - nothing OSX-specific about it.

Drawing progress bar

It's fairly simple to think of a regexp to parse experience points out of the log - the only nontrivial bit is skipping rules section. But we'd much rather have pretty icon than numbers.

Well, first let's parse XP and convert it to levels, in a typical quadratic system (code assumes Symbol#to_proc; upgrade your Ruby to 1.8.7 or paste it from core_ext if it dies).

module XP
  class << self
    def read_xp
      log = File.read("/home/taw/all/xp/xp.txt").sub(/\A.*?= \d{4}-\d{2}-\d{2} \S{3} =\n/m, "")
      log.scan(/^\+\d+/).map(&:to_i).inject(&:+)
    end
    def threshold(level)
      10 * (level ** 2)
    end
    def stats(xp=(ARGV[0] || read_xp).to_i)
      level = 0
      level += 1 while xp >= threshold(level)
      [xp, level, xp-threshold(level-1), threshold(level)-threshold(level-1)]
    end
  end
end

XP.stats returns an array of [experience ponits, level, XP since acquiring current level, XP needed from current level to next] so code drawing pictures doesn't have to think about it.

Now the drawing part. We want 32x16 icon - height is constrained by OSX menu bar height, this ratio just looks good. On the bottom we have progress bar, on the top 1-4 colored circles showing your level - levels 1-4 are green, 5-8 are cyan etc. Here are some examples:

And the code:

require "rubygems"
require 'RMagick'

canvas = Magick::Image.new(32, 16){
  self.background_color = 'transparent'
}
Magick::Draw.new.instance_eval {
  stroke('black')
  fill('none')
  rectangle(2, 9,  30, 14)

  xp, level, xp_got, xp_needed = XP.stats

  stroke('none')
  fill('#0F0')
  rectangle(3, 10, 3 + (29-3)*xp_got/xp_needed, 13)

  level_colors = %w[#0FF #FF0 #F0F #00F]
  while level > 4 and !level_colors.empty?
    fill(level_colors.shift)
    level -= 4
  end
  level.times{|i|
    x = 4+8*i
    circle(x, 4, x, 1)
  }
  draw(canvas)
}
canvas.write(ARGV[0] ? "xp-#{ARGV[0]}.png" : "/home/taw/all/xp/xp.png")

In both parts of the code weird ARGV[0] code is just there for making sample images instead of reading the actual log. Not exactly a masterpiece of modularity, but that's real world Unix scripting for you ;-)

Putting it in OSX menu bar

All this work is fairly useless we put the icons generated somewhere no display - like on OSX menu bar. So far everything has been reasonable system-independent - this part is highly OS-specific - unfortunately all cross-platform GUI toolkits take lowest common denominator approach, and look like shit on all systems equally. But it's really simple code. It's really easy to make the menu icon interactive but then I don't have any particular need for it to be interactive yet.

It must use OSX-shipped ruby interpretter (/usr/bin/ruby), not one from MacPorts unless you want to deal with some serious library hell.

#!/usr/bin/ruby

require "osx/cocoa"
include OSX

class Timer
  def init
    tick
  end
  def tick
    system "./draw_xp.rb"
    $statusitem.setImage(NSImage.alloc().initWithContentsOfFile("/home/taw/all/xp/xp.png"))
  end
end

app = NSApplication.sharedApplication 
$statusitem = NSStatusBar.systemStatusBar().statusItemWithLength(NSVariableStatusItemLength)
NSTimer.scheduledTimerWithTimeInterval_target_selector_userInfo_repeats(1.0, Timer.new, 'tick:', nil, true)
app.run 

Alternatives

One thing I found while writing this blog is Chore Wars, which seems to be like a website for getting XP for performing real life actions - there might even be a Firefox port for constant visibility, but making it public is kinda creepy. You don't want Belarusian secret police finding out about your "+50 sent information about Lukashenko's secret Swiss bank account to WikiLeaks".

Small tips for making Unix programming nicer

Every Unix programmer - and I tell it authoritatively based on my statistically significant sample of 1 - creates millions of tiny scripts that he never bothers publishing. Because they're small, so overhead of cleaning up, documenting, and publishing such script would be immense. And not obviously googleable/bingable/baiduable or whatever people call it these days - it might surprise you but most things in the world are not described by a few well-defined key phrases.


I decided to get a few of such scripts, and throw a bunch of tips on top of them - hopefully you'll find a useful trick or two here.

How to avoid Unix destroying your files

First, we need to fix some of the famous Unix brain damage - cp, mv, and bash overwriting your files.
Now a case could be made for rm removing files without asking for confirmation - that's what rm stands for.
But how in the world is this:

mv most_awesome_song_ever.mp3 ~/Music/

supposed to quietly overwrite existing ~/Music/most_awesome_song_ever.mp3?
In 99% of cases this is behaviour you do not want - and if you actually do,
you know to type this instead:

mv -f most_awesome_song_ever.mp3 ~/Music/

So open your ~/.bashrc and put these commands there (at least ones for mv and cp):

alias rm='rm -i'
alias mv='mv -i'
alias cp='cp -i'

There's one more way Unix can destroy your files - command >file redirection will overwrite file without asking, so it's pretty easy for accidents this way. You really don't want

cat >~/notes/girlfriend

to lose information about all your exes, do you? (example purely speculative) Get yourself into habit of always saying command >>file - appending to file. 99% of time the file in question won't exist - so the result will be the same - and in cases where file exists, you remove it first. If you make a mistake, you just ^C and you're back when you started. I haven't used overwriting redirection is years - it's completely purged from my Unix dictionary. Do the same thing.

By the way some Unix distributions do these by default. And some shells have options for asking for confirmation if > would override files. Do these anyway, just to be sure.

Make shell history useful

By default shell history stores entire 500 entries - a sensible decision back when you had 4MB of RAM, most of which taken by Emacs. It's just ridiculous these days so as first line of .bashrc put export HISTSIZE=1000000. It must be first, because if bash ever decides to exit without HISTSIZE set to a sane value, it might decide to trim your history file and lose all your history. Of course if you want to tempt the fate...

Get rid of .pyc file everywhere

Python's habit of creating .pyc files all over the place gets on my nerves - and it's dubious that they really improve performance that much. If they want to fix performance, how about dealing with GIL first instead of resorting to such hacks? Anyway export PYTHONDONTWRITEBYTECODE=1 totally fixes the problem without any side effects.

Saner output from Unix commands

Compared to fatal loss of data, these are just minor annoyances, but they're really easy to fix, so let's do it.
For stupid reasons du and df commands give sizes in units of 512 bytes or something like that.
Probably some ancient BSD file system allocated files in multiplies of that. That they care more about file system
implementation details than about human usability tells you something about the Unix mindset (if these were at least kBs
that would make sense but no...) - and in any case these assumptions are no longer true on modern operating systems,
which don't rely so mindlessly on blocks. So add these two lines to get human-readable output:

alias df='df -h'
alias du='du -h'

By the way alias command only applies to what you type in the terminal, not what scripts do. So if you run mv or df from scripts, it will not have this default -i/-h. Be cautious.

Final touches for .bashrc

If some commands require root access, and you're tired of typing sudo this and sudo that, just add a few of:

alias port='sudo port'
alias gem='sudo gem'
alias apt-get='sudo apt-get'
alias reboot='sudo reboot'

to your .bashrc. They only save you a little typing, and don't chance anything about security (you still need your admin password etc.), but why type more if you can type less.

Colorful shell

Depending on your distribution you might already have colors in your shell or not.
export CLICOLOR=1 in .bashrc will convince many commands that you want nicely colored output.
To tell that to git, you need to add to your .gitconfig:

[color]
 diff = auto
 status = auto
 branch = auto

And now your Unix comes with more rainbows.

GNU grep must die

Never use GNU grep. Use pcregrep for everything - or rak/ack if you want coloring, automatic .git directory skipping etc. Unfortunately pcregrep is ridiculously slow to type so do yourself a favour and add

alias gr='pcregrep'

to make your life even easier.

wget HTTPS nonsense

I don't know if it's just MacPorts' version of wget, or is it universal, but it seems to miss all HTTPS root CAs (take that VeriSign!). Another alias solves the problem. By "solves" I mean it opens a massive security vulnerability, but we already know that CAs will create fake certs for NSA, Mossad, RIAA, and Hackney Borough Council if asked, so the vulnerability is much less than it seems at first.

alias wget='wget --no-check-certificate'

Use ruby or perl for nontrivial actions

In early 1980s, before Perl got invented to solve exactly this problem I talk about,
people would write insanely complicated shell scripts to automate their Unix actions.

Unfortunately you cannot really serve two aims at once - being highly accepting for casual real time input,
and being highly robust for programmable interfaces - so shell sucks at both. Fortunately the problem
is solved since December 18, 1987 when Perl got invented just for this reason.

And yet - many people act as if 12/18 never happened. Wake up sheeple!

Shell is ridiculously stupid. It has arcane and fragile escaping rules.
It cannot even reliably expand a file list:

$ ls ~/porn/*.jpg
-bash: /bin/ls: Argument list too long

How useless is that? And don't even get me started with xargs, find, awk and the rest horrible mess.

How do you find top largest MP3 files - in any subdirectory?

ruby -e 'Dir["**/*.mp3"].sort_by{|fn| -File.size(fn)}[0,10].each{|fn| system "ls", "-l", fn}'

And yes, I'm calling ls 10 times here, as otherwise it would see fit to rearrange them alphabetically just for lulz.
This Ruby code is really easy and really obvious.

What would be shell solution? Something like this:

find . -name '*.mp3' -print0 | xargs -0  ls -l | sort -k25 -rn | head -n 10

You need find and xargs to avoid "argument list too long" error, then you need -print0 and -0 because file names can contain single quotes or - longcat forbid - spaces! Then you need to manually count at which position of ls -l's output is file name (conveniently ls -l uses something close enough to fixed column width to make sort work - otherwise you'd have to do some heavy awking around it), then you finally head. Personally I prefer waterboarding to having my brain suffer any of this.

Learn GUI integration basics

The chasm between nice programmable world of Unix terminals and hostile world of closed GUI programs can be to some extend lessened.

If you use KDE, dcop command gives you decent level of control over GUI programs and you can explore the interface from command line.

On OSX there's a convenient open command for opening URLs and files with the most sensible program. And osascript command, which is about as powerful as KDE's dcop except far more painful to use by trying to be "friendly" too hard, resulting in unsurprising failure. Google will help.

Editing your scripts made easy

You probably have five billion scripts in your path (you have ~/bin or ~/local/bin or ~/gitrepo/bin or such in your $PATH right?) - and you're tweaking them all the time.
Typing mate `which some_script.rb` takes forever and is not easily tabbable (Ubuntu has really good
bash autocompletion package which might alleviate this problem a lot - but most distros don't).

Wouldn't it be easier to just say some_script.rb --edit? It would also be far easier to type - somTAB --edit. It's really easy. Just put this below the shebang line of all your scripts:

# Ruby
exec 'mate', __FILE__ if ARGV[0] == '--edit'

# Perl
exec "mate", __FILE__ if $ARGV[0] eq '--edit';

# Python
import os, sys
if sys.argv[1:2] == ['--edit']: os.execlp("mate", "mate", __file__)

If someone passed --edit as first argument it will start the editor instead of running the script - otherwise it will not affect it in any way.

Python code is fairly painful because Python decided to keep low level C interface to exec* instead of providing sane Perl-style interface. And you know something is wrong if Perl is described as "sane" compared with you.

Feel free to figure out how to get this effect with C++.

Find kittens for your blog

I only want CC kittens, so nobody sues my blog. Except for defamation, that I don't mind. Here's the script

#!/usr/bin/env ruby
uri = "http://www.flickr.com/search/?q=#{ARGV.join '+'}&l=cc&ss=2&ct=0&mt=all&adv=1&s=int"
# On OSX
system "open", uri
# On Linux
#system "firefox", "-new-tab", uri

By the way could someone get Linux distros to copy open command? It's really simple and really useful.

That's it for today. Enjoy your Unix.

How to be a high-status blogger (like Robin Hanson)

Welcome my to highly inflammatory and highly meta post, which I wrote to show-off my high status as a blogger, obviously. I could have made this post more structured and logical, but it would suggest I actually care about my readers, and caring is low status, right? Or is it? (making this sound deeper will definitely make the subject more high status).

So what's Robin Hanson's theory of status? The basics are simple - people have different hierarchical "status" in society, which is based on organization of paleolithic tribes of hunter-gatherers, and before that of primate bands. These monkeys/cavemen/or such would always try to figure out what's whose status in the tribe so they could engage in tribal politics effectively, but people are not cats so they don't come with captions (making silly jokes is high status, I think), so they had to rely on subtle cues - minor differences in behavior between high-status and low-status monkeys. If you acted like a high-status monkey, everyone would assume you're a high-status monkey and treat you accordingly.

Funnily enough, there are even some experiment showing such effects in monkeys. (I could find you a Wikipedia link, but I already told you what caring would signal) The difference is that with monkeys it's pretty easy to figure out what high-status or low-status behavior would be - and quite easy for researchers to fake it - so it's nice testable theory.

But that's not what Hansonian status is about. Meta time! People incorrectly assume that bloggers write because they believe what they're writing to be true! By showing a few examples of popular and yet false blogs <GLaDOS>insert link to a study showing this here</GLaDOS>, we can convincingly demonstrate that bloggers don't write things that are true, but to signal their high status. It doesn't really follow? Well, it doesn't matter - it works well enough for showing off you're smart and high-status, and that's the only thing which counts in life!

That's basically the methodology. And it's not just Robin Hanson, I've seen far too many people on Less Wrong doing the same - without any doubt they're just trying to associate with high status originator this way.

And what are these high and low status signals? Now it gets even more interesting:

• Anything for which you can make a just-so story that it might have been a high/low status signal in some imaginary primate/Neolithic community, can be treated as high/low status signal without any further evidence
• Anything which some of people you think are high/low status today seem to be doing, can be treated as high/low status signal, or not, depending on how you feel about it, and regardless of there existing even a correlation.
• Anything that seems high status might be treated as low status, because obviously only low status person would try so hard
• Anything that seems low status might be treated as high status, because obviously only high status person could signal that they don't care what others think like that

That is - virtually any behavior can be described as high or low status, depending on how you're feeling on a particular day. Mocking others' theories like that is obviously high status, as some high status people like... let's say Jon Stewart do so. And I don't have to care if other high status people like let's say The Pope don't - or if it might be as common or more among the low status people - such concerns are no part of the Hansonian methodology.

This all has zero predictive power, and no falsifiability whatsoever.

And it doesn't have to be that way. Status theory has a revival after being long forgotten, but it started in late 19th century with Thorstein Veblen's The Theory of the Leisure Class. Now I wouldn't advice reading that book - it's the most painful kind of 19th century literature - unfortunately I know of no better more modern versions. (see what I did here? I'm showing off that I've very knowledgeable and can do impressive things like reading difficult books - so high status!)

Veblen's theory - while based on about as much nonsense just-so stories explaining how status came about - was much simpler than Hanson's, and had far more predictive power. Signals according to it are:

• Spending money on shit you don't need that everyone can see (conspicuous consumption) - high status
• Wasting time on useless activities in a way that everyone can see (conspicuous leisure) - high status
• Doing real hard work - low status, and the more real and harder it is, the lower the status (Veblen didn't believe any of business / management really counts as useful work, and I can definitely see his point)

That's about it.

This can actually predict something, right? Look what kind of signals different ways to spend a day would show:

• Flipping burgers at MacDonalds - really low status, you pollute yourself with menial work
• Playing Excel accountancy games at the office - still low status, at least you avoid menial work pollution
• Playing video games - you waste time but mostly privately, so it counts for little; doesn't show you have money to waste - so still pretty low status
• Going for holidays in some popular destination - shows you have some time to waste, and some money to waste - higher status
• Going for really expensive holidays like on the Lower Earth Orbit - definitely high status, but could use some more conspicuous leisure
• Holidays excavating Mayan ruins - not only shows you have loads of money, it also shows you had ridiculous amounts of time to waste to learn something as useless as Mayan archeology - very high status

That's Veblenian logic. Of course there's no research proving any of that, but who cares. At least it predicts something. Hansonian theory could make a just-so story that burger flipping is high status as it shows you control access to food, which must have been very high status in Paleolithic tribal society - just imagine that - someone who could provide the entire tribe with BigMacs would be the instant alpha male of the tribe (not that such wolf terminology has any applicability to human societies, but anyways)! Or alternatively burger flipping might have Hansonian high status by showing that you so totally don't give a shit what others think about you, you're going to flip some burgers, oh yeah! Unfortunately predictive power and falsifiability don't correspond with high status in either of theories (but feel free to make some just-so stories showing that they do).

Now if you think about it - writing such post shows off that I have plenty of time to publicly waste. Which is appropriately meta.

Please privatise Royal Mail already

I woke up today, and on my way out checked the physical mailbox - and I wasn't even that surprised any more that it contained a "Sorry, you were out" note from the Royal Mail. It's probably the fifth or so time when the Royal Mail cannot find me when I'm at home. Now think which of these is more plausible:

• Aliens keep abducting me for anal probe experimentation every time Royal Mail is supposed to deliver a package for me.
• Those lazy unionized asshats don't even bother ringing and just add the note to the pile of letters.

And no - it's really impossible to miss the doorbell, it's ridiculously loud and annoying, so that's not it. Now let's think for a moment - what would a reasonable package delivery policy be for cases where the recipient is missing - not to mention cases when your staff is too lazy to even bother checking that?

1. Try delivering the second time
2. Have the package available for pickup from the nearest Post Office
3. Have the package available for pickup from some Delivery Office in the middle of nowhere, opened at highly inconvenient hours

You can probably guess when I'm going. And nobody else - none of the private package delivery companies, or fast food delivery guys, or Tesco grocery delivery service, not to mention normal guest - ever had such problems - it is only the Royal Mail and they keep failing over and over again!

So what should be done about it? Privatize their lazy asses, that's what! Maybe it would help, maybe it wouldn't - but right now these fuckers not only cannot deliver mail, they suffer from a severe case of Royal Mail Entitlement Syndrome - and privatizations are a really good cure for that.

Hopefully that will be one good thing that will come out of Tories taking over the country. Labour became so much like Tories already anyway that the only difference between them is Labour's testicular deficiency when it comes to dealing with unions.

The real reason behind Greek economic problems

According to CIA, right now Greece has GDP of $405.7bln, and debt of $552.8bln (136%) - not even counting debt hidden thanks to Goldman Sachs shenanigans. It got into massive debt by spending too much and taxing too little. But everyone seems to forget what it spent so much money on. Here's the real reason:

Greece spends far higher percentage of GDP on military than any EU country; higher than US, Russia, and any even remotely sane country in the world.

Greece spends completely insane 4.3% of its GDP on military - that not even counting forced unpaid labour system - if every soldier was actually paid the number would be even higher - and it definitely severely damages economy by taking people at their most productive age away from useful activities like studying or working.

And what's their track record?

• They obviously lost during the Second World War, but so did everyone else, and then they proceeded to...
• Have a Civil War 1944-1949! The first major success of Greek military was murdering fellow Greeks, with British and American assistance.
• The Civil War being won, the army still felt like shooting some people but didn't have balls enough to face another real army, so not soon later, in 1967, they overthrew the government to kill even more Greeks (they especially loved killing students). With CIA involvement of course, like most Cold War right wing military coups.
• What did the military junta do after they ran out of people to kill in Greece? By sponsoring a military coup attempt in Cyprus with annexation of it into Greece as the ultimate goal. That failed spectacularly, as Turkey would have none of it, and moved its army to annex Northern Cyprus. The fiasco caused collapse of Greek military regime.
• So what did Greeks do after that? Dissolve or at least drastically reduce the army whose only occupation was shooting at their compatriots and unsuccessfully trying to occupy other countries? Of course not - they kept spending insane amount of money and forced unpaid labour on it!

Let's look at list of countries by military expenditure, some notable cases.

• Greece - 4.3% - way too fucking high, never won a war, killed fuckloads of Greeks
• USA - 4.06% - while being involved in countless genuine wars
• Russian - 3.9% - you're clearly spending far too much if even Russia is saner than you
• South Korea - 2.7% - and they're next to a nuclear-armed madmen just waiting for the best time to hit them
• France - 2.6% - second highest in EU, manage to have built third largest nuclear arsenal for that
• UK - 2.4% - and that includes nuclear arsenal and they even manage to win a war every now and then without American support
• Taiwan - 2.2% - they clearly don't care about Chinese
• EU average - only 1.69%
• People's Republic of China - 1.7% - they're clearly all hippie peaceniks!
• Japan - 0.8% - being in reach of North Koreans nukes? We just love our animes and do not care about such things!

There is absolutely no reason why the miserable failure that Greek military is should cost anywhere near as much as it does. And don't even try to mention of Turkey - Greece is part of NATO and has all the NATO guarantees; Turkey was never remotely interested in invading Greece; and their Greek-military-initiated confrontation over Cyprus already resulted in massive failure of Greek military.

Imagine that instead of insane 4.3% of GDP Greece spent only 1.7% of GDP in it, like a typical EU country. Over the last 20 years (before that there was Cold War, which was seriously overrated but let's excuse that), it would save them 52% of GDP - moving them from the list of bankrupt countries to the list of fairly average countries struggling with the recession. But there's more! All that 52% of GDP incurred compound interest. Average long term interest rates are about 6%/year nominal, so 1bln drachmas spent in 1990 grew all the way up to 3.2bln drachmas now (or however many euros that is). Even worse - countries which are deeper in debt have to pay much higher interest rates.

Now exact calculations would require far too much data gathering, so I won't bother at the moment, but one thing is clear even from this:

The only reason Greece is in such crisis now is because they have ridiculously oversized army. The main requirement of any bailout package should be massive reduction in size and cost of their armed forces.

Evolution of the Total War series

Yay, another post about Empire Total War, plus some Rome and Medieval 2! In case you're wondering - I don't intend to even look at Napoleon Total War for let's say another year or two - that's because the first rule of Total War is:

Initial releases of all Total War games are bug-infested piles of shit. Patches fix most of that.

That is, the versions which reviewers fap over so hard are worst.

So you might wonder - why do I even bother playing then? Well, the thing is video game industry like all content producing industries is a oligopoly - there's a really small number of video games made, so the choice is between highly flawed ones and finding another hobby. Like one which might involve going outside more... Let's better not think about it too much. And as I said - patches fix most of the most atrocious problems.

I have no idea what were the main problems of 1.0 ETW. By the time ETW came out I was smart enough to not play it for a year or so; I've only heard all users universally complaining how bad it was. Which is highly believable considering how atrocious M2TW 1.0 was, and about half of these bugs were fixed in vanilla patches (and not even all in mods):

• cavalry couldn't charge (pathfinding issues?)
• pikemen preferred their crappy swords to pikes
• soldiers with two-handed weapons would just stand around confused (attacks in M2TW were based on animation system, and they lacked sufficient animations; RTW just ignored animations so had no such problem)
• having shields made units much worse at defending than not having shields (animation again?)
• ballista/cannon towers didn't shoot what they were supposed to
• during sieges you couldn't move your armies into many large empty places
• treating civilians well gave you negative reputation

And these are just outright bugs, not minor issues like crappy AI, lack of balance, or far too much micromanagement...

Improvements

Anyway, let's focus on what was improved from (patched) M2TW to (patched) ETW. Easily the biggest improvement is huge reduction in micromanagement.

• There are no diplomats/princesses at all. You can talk with anyone you want. RTW diplomats were at least useful for bribery, M2TW made it pretty much impossible.
• Map is all discovered. Fog of war only affects enemy units.
• Units move a lot faster. Especially on roads and by ships, difference between road and wilderness speed is much greater than in M2TW.
• There are decent free garrisons for all cities, so you don't need to keep so many goddamn armies everywhere.
• There is no agent recruitment, all agents spawn automatically.
• There are no annoying random rebels spawning 3 units of Peasants somewhere where it wastes you 3 turns to move, win without a single soldier lost, and waste 3 turns to move back.  Rebellions are very rare, based on population happiness, and fairly high profile when they happen.
• Of the agents left, merchants were replaced by trade ships which move very fast, and provide big money, working in stacks - far less micromanagement again.
• Priests were replaced by missionaries, who are much fewer in numbers and more effective (at least in Americas).
• Spies/Assassins were replaced by Rakes, which are of quite dubious use. M2TW spies were overpowered because they could open settlement gates; while assassins were outright useless. Rakes are just plain useless. You lose diplomatic point, and it costs your enemy almost nothing to repair sabotaged buildings or autorespawn assassinated agents. At least they move fast.
• The last agent type - Gentlemen - is a pretty stupid idea. They mostly sit around in schools, passively adding research points. They can duel - which is completely useless due to autorespawning; and steal research - which I guess is marginally useful, but not sufficiently to warrant another agent type.
• Per-settlement single tax replaced by per-theatre two tax rates plus per-settlement tax exemptions. Mathematically you have fewer choices now when you have at least 5 settlements per theatre, so it's about even... in practice it seems to be somewhat less work.

As a result you can realistically finish the campaign without having to waste far too much effort on campaign micro, and deciding it would be more fun to start over from zero like you did in RTW and M2TW. Now campaign mode is in no way perfect, but the improvement is so huge... they finally got their act together and fixed one of the biggest problems plaguing the series.

There are other improvements. If you include both faction differences and research level differences, there's pretty decent unit variety.

As bad as ever

Plenty of problems from earlier TW games still affect ETW. Campaign AI is still crap. I'm not sure if it improved at all - it keeps doing ridiculous things like attacking me first with half its units, then again same turn with the second half - because merging before attack is clearly too hard. It has absolutely no idea how to do a naval invasion - which sort of makes is unable to organize any attack larger than pirate raids now that oceans became so important.

Battle AI is even worse. The simplest, dumbest formation you can use is just form a lot of Line Infantry units in a long shallow line, facing enemy. And this seems to be able to enough to destroy almost any AI army twice its size with hardly any loses. Instead of flanking, or at least atacking en masse, AI insists on running to center of my line with 1-2 units at time, and not even spread out, so they get 200 of my guys shooting at their 10. To entirely crush AI I can fold my flanks into a horse-shoe - what wouldn't be possible if AI bothered to attack my flanks instead of storming the center.

A bunch of Line Infantry with one howitzer is enough to take any fort - you need to approach diagonally, as there's a zone where fort cannons cannot fire; and then a bit of howitzer fire is enough to make AI units leave the fort and attack you - and again 1-2 units at a time. Carcass shot is needed if your howitzer is to cause any kills, but vast majority of their soldiers are going to die running into your line infantry alone.

Battle pathfinding is as bad as ever. All places which seem empty and available are now at least empty and available - but it's very common for whole unit of Line Infantry to stand idle and wait for one guy trapped on some obstacle instead of fucking him and firing at the enemy.

Units also seem to ignore orders a lot. Not even complex orders - I've seen artillery keep firing after I explicitly told it to stop more than once. When they decide to fire, they often fire at something else than I ordered, or even right at my units for no obvious reason. Fire-at-will button is basically a license to ignore your targets (the same was true with M2TW).

And while units usually behave reasonably well on an open field, once they get into a fort fight they get so unresponsive than anything more complicated than a bayonet charge is probably not going to work. Fortunately AI is usually kind enough to loss 80% of its units while charging at my infantry line before I even get to the fort, so bayonet charge might as well finish the job.

Regressions

Unfortunately there are some ways ETW is significantly worse than RTW/M2TW. Naval battles for one. They were just a huge mistake, they're not fun, they don't add anything to game, fortunately you can still do the RTW/M2TW thing and click autoresolve.

Another problem is tile-less campaign map. Figuring out if unit can reach some place, or merging units gets a lot more finicky. Every time I want to do any non-trivial campaign maneuvering, I save the game - the chance is good something really stupid will happen. This is the only part of campaign mode which I would really rather see reversed.

Empire crashes to desktop a lot more often than RTW/M2TW. Even the patched version. I just had a campaign as United Provinces which ended with me having 39 regions, and the game reliably crashing every time I click end of turn, just after last enemy faction moves.  I loaded the savegame and bought the final province from Ottomans for an easy victory, but that really shouldn't be happening.

Another big problem is total lack of documentation. There's no in-game documentation, no paper (ha, Steam) or PDF tutorial, just absolutely nothing for plenty of game functionality. Do you know what are pros and cons of various government types? I've seen many mutually conflicting answers to that online.

And the worst problem of all - moddability really suffered. RTW kept everything in plain text files, so modding was quite easy even if they never really gave it much thought. M2TW moved files to packs, ETW makes it even harder... And unfortunately it's not just their laziness - they figured out that if they make modding harder, they can sell DLCs with extra units / campaigns etc. - things modders would release in buckets, much more fun and much higher quality - if only they had access to proper modding tools. And they still sort of can, but with more and more effort with each game, and I'm afraid Creative Assembly will eventually kill the modding community.

There are fewer battle maps now for some reason - it doesn't strike me as terribly much effort to make them, so why wouldn't they? Custom battles seem more limited - you cannot select technology levels etc. for different sides... in previous games you could have pretty much any battle in custom mode that was possible in campaign mode.


There's one more thing about campaign mode which is really puzzling, by which I mean retarded. When you get required number of provinces, let's say 40, by 1730 - you still have to wait until 1799 to officially "win" the campaign. So 2/3 of the game time spent doing absolutely nothing at all? I don't see how anybody could think it could be a good idea.

And as I said in previous post, user interface for battles got significantly less readable for some mysterious reason.

What next?

Now tasks for my readers: have any of my complaints been addressed by either Napoleon Total War or mods? Some seem fundamental, but others seem fairly trivial so they might have been fixed already, right?