Efficient go hypothesis

Efficient market hypothesis states that you cannot really beat the market by much, all the prices are about right, and everybody behaves it more or less rational way. Yes, treating it too seriously leads to ketchup economics thinking, but it's often a good starting point - if you postulate that some prices are seriously wrong, you better come up with a story why they're wrong.

But let's forget about economics for a moment. Something very similar to efficient-market hypothesis applies to the game of go. Unlike most other games like chess and football, go is extremely balanced.

• Black player, who moves first, pays 6.5 points for the privilege, what makes both players' chances of winning as close to 50:50 as it gets.
  
• If one player is stronger, the weaker player gets as many free stones as necessary to get the chance of winning very close to 50:50.

What's interesting is that during the game, it's impossible to make a move that highly increases your advantage, or "beats the market" - if you could get significantly ahead by playing a stone, then the position was already unbalanced before you've done so due to this possibility. Values of positions consist of complex balance of territory, influence, flexibility, good and bad aji, and many other things - if you ask a computer program like gnugo to estimate balance of the board, you will get widely varying numbers with each moves, each totally ridiculous.

Just like on the market we cannot give correct formula for a price, but we know it's right because people trade at it; in go we don't have a formula for value of a position, but we know it's even if good moves by two players led there. If you postulate imbalance, you better come up with a story telling which move was bad and why.

Go players often perform some complex trades of moves called joseki - standard sequences that lead to more or less even results. Again, why even? Because neither you nor your opponent would purposefully play bad moves to fall behind, and if there are no good moves available it means one of you was already losing earlier.

Joseki example - every single move here and the sequence are about even

This requires significant shift in thinking compared to games where you act to get as far ahead of your opponent. You can play go like an index fund - just try to stay even with each move, and let your opponent make some mistakes and fall behind you.

That doesn't mean there's no room for creativity - sometimes there's only one good move to make, most of the time you have multiple good options with subtly different balances. These tiny variations are enough to win - there is no possibility of draw in go (not counting super-ko and such exceptional situations), so even if you get just 1 point ahead through the entire game, 0.01 point advantage per move, that's just enough. It's kind of like high frequency trades, which earn tiny profits, but with sufficient volume it's worth it.

Most professional games end with just a few points advantage for one side or the other. Or one of the players falls behind just a bit, and then risks it all on a great all-board fight to avoid the few point loss.

So as you can see, go is just like market. Except it never leads to crises costing trillions of dollars.

Get better at go with gnugo

Usually I write about things that work quite well - who wants to learn about stuff that doesn't? Or about spectacular failures. Near misses - things that almost work but not really - are a rare subject. This post is about one of such near misses.

I'm sometimes casually playing go at KGS. My account is at 10k now, not too impressive, but then I don't play that much. That's just what my personality is like - I don't do that many things with any regularity, I start something, get really into it, and then a few weeks later my enthusiasm wanes. Not always, but it's quite typical.

So of course I would like to get better at go, without too much effort if possible. I know what's your first thought, but it's not about not losing. You know what's the biggest problem with multiplayer games? For every winner there's a loser, and 50% of people must lose, and the winners tend to be asshats without life who spend all night playing the game, making it impossible for casual players to get anywhere - sure, a casual player like me could slowly learn how to play in multiplayer mode, but losing to asshats a few times too many tends to be too discouraging.

So go is not like that. Everyone has rank assigned, and when people with different ranks are playing the weaker player gets big head start. If you win too much, your rank increases. If you lose too much, your rank decreases. So typically everyone wins half of the time, and loses half of the time - unusually convenient situation for casual players. My rate of wins and loses is pretty much independent of how good I am.

If that's the case, why am I interested in getting better? Mostly because I really hate losing due to spectacularly bad moves. Sure, there are legitimate ways of losing the game - playing too slowly and not getting enough territory, losing a big fight, the normal cases. What is not cool is losing the game by moral equivalent of putting yourself in atari, like this 9dan pro did:



How can one get better at go? First one can play a lot. That definitely helps, but there are quite a few permanoobs - players with way too many games that never progress much. And just by playing and not doing anything else it would take ages to get better.

The second way is doing go problems. They let you get better at fighting and endgame, maybe teach you some common joseki - all very useful things, but they won't teach you any big concepts, or improve your grand strategy. I've seen quite a few players who could win every fight with me, and then lose 20 points by playing some completely useless move, many times during one games.

The third thing is serious studying, like watching games of pros and good amateurs, reading Sensei's Library and books etc. I tried that, but I don't really see what's going on in pro games, and Sensei's Library is mostly targeted at dan players, so it's of limited use to me. And of course it takes a lot of time.

The fourth way, and one that I really like, is reviewing your games. You can review alone or with your fellow player - especially useful in handicap games where White can teach Black something, no matter who won. Or you can send your game to The Go Teaching Ladder, to get a review by a stronger player. Unfortunately that takes ages, and you cannot review every single game this way.

So I thought - bots aren't that horrible at go, some are near shodan level, why not use bots to do a quick review? Now certainly, it won't be as good as a human review, but it's really quick, and hopefully would be of some value.

Gnugo already has some analysis mode - it can take sgf and annotate each step with its idea of what possible moves for Black and White there are and how good they are.

Because its interface is quite inconvenient, I wrote this tiny Ruby script, so I can say annotate_sgf foo.sgf to get annotated-foo.sgf:

level = 15
ARGV.each{|sgf|
  annotated_sgf = File.dirname(sgf) + "/" + "annotated-" + File.basename(sgf)
  system 'gnugo', '--level', "#{level}", '--output-flags', 'dv',
         '--replay', 'both', '-l', sgf, '-o', annotated_sgf
}

And here the problems start. First, while top bots are around shodan level, gnugo is not. It's more like 8k, not horrible (I wouldn't mind a review by an 8k player), but not impressive either. Especially for weak double kyu players it might be pretty attractive.

What gnugo is pretty decent about is noticing groups that were unsettled but both players forgot about them.



Here both players ignore T16, which is obviously vital point for White group if you look at it. I haven't bothered thinking if S6 is another such point, but if gnugo think so, that's at least reason enough to check.



Here's another example - players didn't notice that Black had a ko to mess with White. Gnugo is pretty decent at endgames, and it's surprising how often double digit kyu, and weaker single digit kyu players make such big mistakes.



Gnugo can also point at players making moves that are too small. N1 is a valid move in endgame, where nothing else interesting is going on. At this point it's just small gote reduction. This is extremely common among weak players - territory is real, and influence is vague, so they're willing to fight for scraps of territory instead of grabbing major influence over the board.



Gnugo can also help you stop making unusual fuseki and joseki moves. Unfortunately it's mostly just applying its joseki dictionary without thinking much, so you must think some more if the move is really bad, or gnugo just doesn't like it. Unlike with endgame and semeai, it's hard for you to verify if gnugo is right.

So why don't I like gnugo in spite of all this? Well, I don't really think I'm learning much from it. It doesn't tell me why a move is good or bad, it doesn't create variations showing what are the consequences of different moves (even though it obviously tries them internally). It might tell you which move lost you the fight if you're really curious, but it's difficult to generalize this to new games.

Basically, I don't think I got much better thanks to gnugo. But try it if you want.