The best kittens, technology, and video games blog in the world.

Saturday, May 12, 2007

Using a DDR dance mat as a musical instrument

The Other Side... by Tim Morgan from flickr (CC-NC-SA)That's an idea I had for a very long time - connect a DDR dance mat to some music generation software so that people would control the music by dancing. Step first is getting data from a dance mat. Most dance mats these days are USB HID devices. Their data can be read from /dev/input/event<number> (generic HID interface) and /dev/input/js<number> (joystick interface).

I wanted to get some clue what the data might look like, so I catted a sample of it to sample file and then reverse engineered the format. I guess there's documentation somewhere, it's just often faster to look at the output and guess what it's supposed to mean.

key = Hash.new{|ht,k| k}.merge({
292 => "up",
288 => "down",
289 => "right",
291 => "left",
290 => "up-right",
293 => "up-left",
295 => "down-right",
294 => "down-left",
297 => "select",
296 => "start",
})

data = File.read("sample")
samples = data.size/16
samples.times{|i|
sample = data[16*i, 16]
sec, ms, type, code, value = sample.unpack("VVvvV")

tm = Time.at(sec).strftime("%H:%M:%S")
ms = sprintf("%06d", ms)
# Types:
# * 0 - SYN
# * 1 - KEY
# * 2 - REL (relative position)
# * 3 - ABS (absolute position)

next unless type == 1
if value == 0
dt = "PRESS #{key[code]}"
elsif value == 1
dt = "RELEASE #{key[code]}"
else
dt = sprintf("UNKNOWN #{type} #{code} #{value}")
end
print "#{tm}.#{ms} - #{dt}\n"
}
So the dance mat driver gives us nicely timestamped key presses and releases. No "simultaneous keypress" events for jumping on two arrows at once, so they would have to be emulated in software. That's about it as far as input is concerned. Now the output.

My first idea was looking for some sort of Ruby (or Perl or whatever) MIDI library. Unfortunately I could find nothing useful. After a long search, I finally got to ChucK, the "Strongly-timed, Concurrent, and On-the-fly
Audio Programming Language". Unfortunately Ubuntu package sucks - it doesn't tell you that you need to install jackd (in Recommended or Suggested or documentation), nor is it in a manual or FAQ or wherever I looked. I found that out by strace and guesswork. So to run ChucK in Ubuntu you need to sudo apt-get install chuck jackd.

Here's the highly uninformative error message you get when you run ChucK without a running jackd:
 $ chuck something.ck
[chuck]: (via rtaudio): no devices found for compiled audio APIs!
[chuck]: cannot initialize audio device (try using --silent/-s)


So open a terminal and start jackd:
$ jackd -d alsa

ChucK still won't work:
 $ chuck something.ck
[chuck]: (via rtaudio): no devices found for given stream parameters:
... RtApiJack: the requested sample rate (44100) is different than the JACK server rate (48000).

[chuck]: cannot initialize audio device (try using --silent/-s)

You need to tell ChucK to use sampling rate of 48000 (telling jackd to use sampling rate of 44100 somehow didn't work). The manual says:
--srateN  Set sampling rate (default to 48000 for jack, auto detected otherwise).

which is doubly untrue, as it used 44100 with jack, and apparently didn't do any autodetection.

After overcoming all the obstacles, finally we can get some sounds.
$ cd /usr/share/doc/chuck/examples/stk/
$ chuck --srate48000 clarinet2.ck
---
reed stiffness: 67.172840
noise gain: 29.734127
vibrato freq: 75.111492
vibrato gain: 72.994233
breath pressure: 121.968367
---
reed stiffness: 120.611360
noise gain: 49.769301
vibrato freq: 60.092423
vibrato gain: 60.048969
breath pressure: 126.965529
^C
[chuck]: cleaning up...


I strongly recommend at this point to try musical version of Towers of Hanoi:
$ cd /usr/share/doc/chuck/examples/
$ chuck --srate48000 hanoi++.ck
move disk from peg 1 -> peg 2
move disk from peg 1 -> peg 3
move disk from peg 2 -> peg 3
move disk from peg 1 -> peg 2
move disk from peg 3 -> peg 1
move disk from peg 3 -> peg 2
move disk from peg 1 -> peg 2
move disk from peg 1 -> peg 3
move disk from peg 2 -> peg 3
move disk from peg 2 -> peg 1
move disk from peg 3 -> peg 1
...

ChucK has C-like syntax and should be reasonably simple to understand. It already has HID examples in /usr/share/doc/chuck/examples/hid/, including some for joystick. After some experimentation, I was able to get them to make DDR dance mat to be a percussion instrument.
Button numbers may be different for your dance mat, but in mine up-left, left, and down-left control three kinds of beats in the left audio channel, and up-right, right, and down-right control the same kinds of beats in the right audio channel. Samples are copied from /usr/share/doc/chuck/examples/data/

// make HidIn and HidMsg
HidIn hi;
HidMsg msg;

// open joystick 0, exit on fail
if( !hi.openJoystick( 0 ) ) me.exit();

<<< "joystick ready", "" >>>;

SndBuf key_snd[8];

// load files
// Left
"data/snare-chili.wav" => key_snd[5].read;
"data/kick.wav" => key_snd[3].read;
"data/snare-hop.wav" => key_snd[6].read;

// Right
"data/snare-chili.wav" => key_snd[2].read;
"data/kick.wav" => key_snd[1].read;
"data/snare-hop.wav" => key_snd[7].read;

key_snd[5] => dac.left;
key_snd[3] => dac.left;
key_snd[6] => dac.left;

key_snd[2] => dac.right;
key_snd[1] => dac.right;
key_snd[7] => dac.right;

// infinite event loop
while( true )
{
// wait on HidIn as event
hi => now;

// messages received
while( hi.recv( msg ) )
{
if(msg.type == 1)
<<< "Press", msg.which >>>;
else if(msg.type == 2)
<<< "Release", msg.which >>>;
if(msg.type == 1)
{
int key;

msg.which => key;
if(key == 1 || key == 2 || key == 3 || key == 5 || key == 6 || key == 7)
{
0 => key_snd[key].pos;
// gain
Math.rand2f( 2.0, 6.0 ) => key_snd[key].gain;
}
}
}
}
Some explaining. It uses joystick-specific interface (/dev/input/js<number>) not generic HID interface (/dev/input/events<number>), so data format and key numbers are a bit different.

source => dest is a connection operator. dac is a sound card, so obviously dac.left and dac.right are its left and right channels. Code like "data/snare-chili.wav" => key_snd[5].read; reads sound files to a buffer, and key_snd[5] => dac.left; plays them (so you get a loud beat when you stard the script). 0 => key_snd[key].pos; resets the position to 0, in effect playing a given beat again. Math.rand2f( 2.0, 6.0 ) => key_snd[key].gain; sets gain (volume) of a particular beat to random number between 2 and 6. <<< ... >>> means print.

The script is just a proof of concept, so I'm not going to upload an mp3 (or a YouTube movie). I think it should be possible to generate some really interesting music by dancing, if the script became smarter. I'd like to hear from anyone who tried to code such a thing.

Problems I had with ChucK were reported to Ubuntu: bug #114161, bug #114162

Syntactic tradeoffs in functional languages

Bear & Avery by `Pacdog from flickr (CC-BY)It's difficult to design a syntax for functions (blocks, closures, anonymous inner classes) as first-class values which looks natural in traditional (aka Algol-like) syntax. There have been three solutions:

  • Use non-traditional syntax - like Scheme, Smalltalk, OCaml, Haskell
  • Accept cumbersome syntax for function values to keep traditional syntax - Python, Java
  • Use traditional syntax, but add some sugar for functions which take just one functional argument - Ruby
Languages from the first group were never too popular, and languages from the second were never too "functional" (I'm not talking about purity, just being able to use high-order functions and stuff naturally). Right now Ruby is the most popular kinda-functional language.

SyntaxNo block argumentsOne block argumentMultiple block arguments
Scheme-likeOKOKOK
Ruby-likeGreatGreatUgly
Python-likeGreatUglyUgly
So Ruby-like syntax is a pure win over Python-like syntax - better for one-block functions, and equally ugly for no-block and multiple-block functions. It is generally considered a win over nontraditional syntax for no-block and one-block functions.

Compared to non-traditional syntax, there's a trade-off. It looks better for no-block and one-block functions, but loses for multiple-block functions.

To find out how common functions taking various numbers of blocks are I grepped OCaml library interface files. I took OCaml because as language with non-traditional syntax it wouldn't have anti-functional bias typical in Algol-like languages, and unlike Scheme/Smalltalk/etc. it is statically typed so it wouldn't take too much work to find out how many blocks functions take. SML and Haskell would probably work too here.

I copied /usr/lib/ocaml/3.09.2/ and extracted inferred type information.
$ ruby -e 'Dir["**/*.mli"].each{|fn| system %Q[ocamlc -i #{fn} >#{fn.sub(/\.mli$/,".t")}]}'

I was only interested in functions so I preprocessed it a bit:

$ cat *.t */*.t |
ruby -e 'print STDIN.read.gsub(/(:|->|=)\n\s*/, "\\1 ")' |
gr -v '^\s*(type|mutable|\}|\||module|end|exception|and|")' |
ruby -pe '$_.sub!(/=.*\n/,"")' |
gr -- '->' >FUNCTIONS

$ cat FUNCTIONS | wc -l
2239
$ cat FUNCTIONS | gr '\([^\)]*->[^(]*\)' | wc -l
286
$ cat FUNCTIONS | gr '\([^\)]*->[^(]*\).*\([^\)]*->[^(]*\)' | wc -l
16
So among 2239 functions defined in OCaml standard library (and whatever libraries I have installed), 87.2% take no blocks, 12.1% take one block, and 0.7% take two or more blocks.

This means:
  • Functions which take one block are common enough for Ruby syntax to be a major win over Python syntax
  • Functions which take multiple blocks are so rare that better syntax for them won't make enough different to offset non-traditional syntax in other cases
  • Ruby-style syntax wins (at least unless other languages have macros)
Here are the functions taking multiple blocks found. In camlinternalOO.mli:
  • val make_class : string array -> (table -> Obj.t -> t) -> t * (table -> Obj.t -> t) * (Obj.t -> t) * Obj.t
  • val dummy_class : string * int * int -> t * (table -> Obj.t -> t) * (Obj.t -> t) * Obj.t
In format.mli
  • val set_formatter_output_functions : (string -> int -> int -> unit) -> (unit -> unit) -> unit
  • val get_formatter_output_functions : unit -> (string -> int -> int -> unit) * (unit -> unit)
  • val set_all_formatter_output_functions : out:(string -> int -> int -> unit) -> flush:(unit -> unit) -> newline:(unit -> unit) -> spaces:(int -> unit) -> unit
  • val get_all_formatter_output_functions : unit -> (string -> int -> int -> unit) * (unit -> unit) * (unit -> unit) * (int -> unit)
  • val make_formatter : (string -> int -> int -> unit) -> (unit -> unit) -> formatter
  • val pp_set_formatter_output_functions : formatter -> (string -> int -> int -> unit) -> (unit -> unit) -> unit
  • val pp_get_formatter_output_functions : formatter -> unit -> (string -> int -> int -> unit) * (unit -> unit)
  • val pp_set_all_formatter_output_functions : formatter -> out:(string -> int -> int -> unit) -> flush:(unit -> unit) -> newline:(unit -> unit) -> spaces:(int -> unit) -> unit
  • val pp_get_all_formatter_output_functions : formatter -> unit -> (string -> int -> int -> unit) * (unit -> unit) * (unit -> unit) * (int -> unit)
In printf.mli:
  • external index_of_int : int -> index val scan_format : string -> 'a array -> index -> int -> (index -> string -> int -> 'b) -> (index -> 'c -> 'd -> int -> 'b) -> (index -> 'e -> int -> 'b) -> (index -> int -> 'b) -> (index -> ('f, 'g, 'h, 'i) format4 -> int -> 'b) -> 'b
  • val sub_format : (string -> int) -> (string -> int -> char -> int) -> char -> string -> int -> int
In equeue/equeue.mli:
  • val create : ?string_of_event:('a -> string) -> ('a t -> unit) -> 'a t
In netstring/cgi.mli:
  • val init_mt : (unit -> unit) -> (unit -> unit) -> unit
And the last one was false positive of regular expression ((unit -> (unit -> unit) * (unit -> unit)) -> unit), but I kept it in the statistics, as filtering only false positives and not false negatives would bias the results more than no filtering (and it doesn't change the conclusion).

Perl-compatible regular expressions

In the code above I used gr which isn't a standard Unix command. I'm simply sick of non-Perl-compatible regular expressions. grep should die, die, die horrible death. Unfortunately pcregrep program has annoyingly long name, so I aliased it gr. Do yourself a favour, run:
$ sudo apt-get install pcregrep
$ echo "alias gr=pcregrep" >>~/.bashrc
and never use grep again.

Thursday, May 10, 2007

Gtk DSL for RLisp

“I’m serious, it was this big!” by Kevin Steele from flickr (CC-NC)
I basically agree with Steve Yegge that "w=widget.create; w.set_attribute; c = widget.create; w.add_child(c)" is the most horrible ways of creating UIs. Well, graphical GUI builders generating obfuscated code and printing HTML as raw strings are close in suckiness.

The best I could do were custom XMLs. "Custom" is the point here, it must have application-specific to be of any use, generic toolkit XMLs are too limited. Building GUIs this way still sucked, but considerably less so. Here's a snippet from my old app, something like those single-page-wikis, just local and database-backed:

class PseudoTiddlyBox < Gtk::EventBox
def initialize(title)
super()
@title = title
@links = nil
@text = ""

vbox = $g.parse_subtree("<vbox />")
button_bar_desc = "<text editable='false'><right>"+
" <button cmd='open_all'>Open all</button>"+
" <button cmd='close'>Close</button>"+
"</right></text>"
@button_bar = $g.parse_subtree(button_bar_desc)
@header = $g.parse_subtree("<text bg='#D0D0D0' editable='false'><huge>#{title}</huge></text>")
@body = $g.parse_subtree("<text editable='false' />")

vbox.pack_start(@button_bar, false, false, 0.0)
vbox.pack_start(@header, false, false, 0.0)
vbox.pack_start(@body, false, false, 0.0)

add(vbox)

@button_tags = []
@button_bar.buffer.tag_table.each{|tag|
@button_tags.push(tag) if tag.is_button
}

signal_connect("enter-notify-event") {|w,e|
@button_tags.each{|tag| tag.foreground = 'black'}
false
}
signal_connect("leave-notify-event") {|w,e|
@button_tags.each{|tag| tag.foreground = 'white'}
false
}
# Attach a link/button controller
@button_bar.attach_link_observer {|w,href| open_link(href)}
@body.attach_link_observer {|w,href| open_link(href)}

@button_bar.attach_tag_observer {|tv,tag,new_state|
tag.background = new_state ? '#F0F0F0' : 'white' if tag.is_button
}
@body.attach_tag_observer {}
end
...
end


With custom XML it was possible to simplify building structure considerably, but attaching code to widgets was as hard as ever. XML is a horrible representation of code. It's awesome for data export, and markup, and configuration files, and many other things, just not code. The code should definitely be in some real language. Unfortunately in this case most languages fail miserably. What has the advantages of both programming languages and XML ? Lisp S-expressions obviously !

To get something running, I first coded a trivial GUI hello world in RLisp, Java style:
(ruby-require "gtk2")

(let w [Gtk::Window new "RLisp Gtk Hello World"])
[w border_width= 10]

(let v [Gtk::VBox new])
(let l [Gtk::Label new "Hello, World"])
(let b [Gtk::Button new "Quit"])
[v pack_start l]
[v pack_start b]

[b signal_connect "clicked" & (fn ()
(print "Hello, world!")
[Gtk main_quit]
)]

[w add v]
[w show_all]

[Gtk main]

It's so ugly. Even in such trivial application it takes a lot of thinking to imagine what the result will looks like. And here's the same program, using macro-based DSL:
(gtk Window ("RLisp Gtk Hello World")
id => w
border_width => 10
(gtk VBox ()
(gtk Label ("Hello, World"))
(gtk Button ("Quit")
signal "clicked" => (fn ()
(print "Hello, world!")
[Gtk main_quit]))
)
)
[w show_all]
[Gtk main]

It feels like Zen. It's instantly obvious what the window will look like and even how it will behave. The difference is almost as huge as going from hand-coded C parsers to regular expressions.

The DSL is very simple, it's just one macro (gtk ...). The first argument is widget class, then constructor arguments, then attributes. id => var means local variable var should be bound to the widget (something I wanted to have in XML-based solution, but it was only possible to use global variables this way). signal sig => handler sets callbacks, property => value sets various attributes, and everything else is considered a child to add. (if (foo) (gtk some-widget) (gtk another-widget)) is a perfectly valid child, but in this DSL optional children must go through the usual "widget.add(child)" road.

Here's the DSL definition. It took just 40 minutes to write, what means that either I'm getting better at macros or it's a pretty simple one:
(ruby-require "gtk2")

(defun gtk-attrs (widget args)
(let tmp (gensym))
(defun parse-args (args)
(match args
('id '=> var . rest)
(cons `(let ,var ,tmp) (parse-args rest))
('signal sig '=> handler . rest)
(cons `[,tmp signal_connect ,sig & ,handler] (parse-args rest))
(prop '=> val . rest)
(cons `[,tmp ,["#{prop}=" to_sym] ,val] (parse-args rest))
(child . rest)
(cons `[,tmp add ,child] (parse-args rest))
()
()
x
(raise "Don't know what to do with: #{args}")))
(let ops (parse-args args))
`(do
(let ,tmp ,widget)
,@ops
,tmp)
)

(defmacro gtk (widget constr_args . attributes)
(gtk-attrs `[,["Gtk::#{widget}" to_sym] new ,@constr_args] attributes))

Wednesday, May 09, 2007

Incremental macro building for RLisp

huren...\:D/\:D/\:D/\:D/ by ariffjrs from flickr (CC-NC-SA)RLisp unit testing framework is based on pattern matching. Its core is assert macro which defines a domain specific language for testing:

(require "rlunit.rl")

(test-suite RLUnitExample
(test example
(assert 9 == (+ 2 7))
(assert 2 == (- 6 4))
(assert 2 == (- 6 4) msg: "Subtraction should work")
(assert nil? nil)
(assert 1.99 == 2.02 delta: 0.05 msg: "Deviation is too high")
(assert '() instance_of? Array)
(assert '(foo bar) instance_of? Array)
(assert '(foo bar) kind_of? Enumerable)
(assert () != nil)
(assert #f same: false)
(assert #t same: true)
)
)
The framework had just one tiny problem. In Fabien's words:
What you'd need now is a way to dynamically extend your matches with additional patterns, so that you can plug new kinds of tests without hacking the framework's sources.
And that's pretty much what I did. Patterns are kept in hash table pattern-macros-db. pattern-macro-create creates a new macro, pattern-macro-extend adds new patterns (prepended to the list), and pattern-macro-recompile recompiles the macro.
(let pattern-macros-db (hash))

(defmacro pattern-macro-recompile (name)
`(defmacro ,name args
(match args ,@[pattern-macros-db get name])
)
)

(defmacro pattern-macro-create (name . patterns)
`[pattern-macros-db set ',name ',patterns]
)

(defmacro pattern-macro-extend (name . patterns)
`[pattern-macros-db set ',name
[',patterns + [pattern-macros-db get ',name]]
]
)
Now the definition of assert macro looks like:
(pattern-macro-create assert
('nil? a) `[self assert_nil ,a]
('nil? a 'msg: msg) `[self assert_nil ,a ,msg]

(a '=~ b) `[self assert_match ,b ,a]
(a '=~ b 'msg: msg) `[self assert_match ,b ,a ,msg]

(a '!~ b) `[self assert_no_match ,b ,a]
(a '!~ b 'msg: msg) `[self assert_no_match ,b ,a ,msg]

(a '!= b) `[self assert_not_equal ,b ,a]
(a '!= b 'msg: msg) `[self assert_not_equal ,b ,a ,msg]

(a 'same: b) `[self assert_same ,b ,a]
(a 'same: b 'msg: msg) `[self assert_same ,b ,a ,msg]

(a 'not_same: b) `[self assert_not_same ,b ,a]
(a 'not_same: b 'msg: msg) `[self assert_not_same ,b ,a ,msg]

(a 'kind_of? b) `[self assert_kind_of ,b ,a]
(a 'kind_of? b 'msg: msg) `[self assert_kind_of ,b ,a ,msg]

(a 'instance_of? b) `[self assert_instance_of ,b ,a]
(a 'instance_of? b msg: msg) `[self assert_instance_of ,b ,a ,msg]

('block: blk) `[self assert_block & (fn () ,@blk)]
('block: blk 'msg: msg) `[self assert_block ,msg & (fn () ,@blk)]
('msg: msg 'block: blk) `[self assert_block ,msg & (fn () ,@blk)]

(a '== b 'delta: c) `[self assert_in_delta ,b ,a ,c]
(a '== b 'delta: c
'msg: msg) `[self assert_in_delta ,b ,a ,c ,msg]
(a '== b 'msg: msg
'delta: c) `[self assert_in_delta ,b ,a ,c ,msg]


(a '== b) `[self assert_equal ,b ,a]
(a '== b 'msg: msg) `[self assert_equal ,b ,a ,msg]

(a 'macroexpands: b) `(assert (macroexpand-rec ',a) == ',b)
(a 'macroexpands: b
'msg: msg) `(assert (macroexpand-rec ',a) == ',b msg: ,msg)

(a b) `[self assert_equal ,b ,a]
(a b 'msg: msg) `[self assert_equal ,b ,a ,msg]
(a) `[self assert ,a]
(a 'msg: msg) `[self assert ,a ,msg]

(raise SyntaxError [(cons 'assert args) inspect_lisp])
)
(pattern-macro-recompile assert)

And it can be extended trivially, also allowing recursive definitions:
(pattern-macro-extend assert
(rlisp 'syntax: ruby) `(assert ,rlisp == (ruby-eval ,ruby))
(rlisp 'not_syntax: ruby) `(assert ,rlisp != (ruby-eval ,ruby))
)
(pattern-macro-recompile assert)

(test-suite Syntax
(test true_class
(assert true syntax: "true")
(assert #t syntax: "true")
(assert 't not_syntax: "true")
)
(test false_class
(assert false syntax: "false")
(assert #f syntax: "false")
(assert nil not_syntax: "false")
(assert () not_syntax: "false")
)
(test nil_class
(assert nil syntax: "nil")
(assert () not_syntax: "nil")
)
; ...
)
Explicit calls to (pattern-macro-recompile ...) are required.
RLisp processes files one global statement at time. So (a) (b) is processed:
  • Macroexpand (a)
  • Execute (a)
  • Macroexpand (b)
  • Execute (b)
On the other hand (do (a) (b)) is processed:
  • Macroexpand (a)
  • Macroexpand (b)
  • Execute (a)
  • Execute (b)
It only makes a difference when execution of (a) affects macroexpansion of (b). What would be the case if one macro tried to expand to (do (update pattern-macros-db ...) (pattern-macros-recompile ...)) - the recompilation would be macroexpanded before pattern-macros-db would get updated. eval would of course work, as would some other tricks, but I think explicit recompilation is not that bad.

Tuesday, May 08, 2007

New parser for RLisp - with string interpolation and regexps

Felipe by José Luna from flickr (CC-NC-SA)
A few hours ago I wrote about syntax supported by RLisp and complained that it lacked decent String interpolation and Regexp syntax.

After writing that I tried to add support for both to the existing ANTLR-based parser. Unfortunately the code generated for parsing interpolated strings raised exceptions instead of backtracking. I think I understand why it was happening, but I didn't see any obvious way of fixing it without making the grammar exceedingly complicated. So I rewrote it all with a bunch of regular expressions for lexer and recursive descend for a parser. Lisp parsers are easy. Now it's possible to write arbitrary RLisp code inside interpolated strings, and write regexps which aren't any uglier than usual:

rlisp> (let m ["40 + 2 = #{(+ 40 2)}" match /(\d+)$/])
#<MatchData:0xb7cacf94>
rlisp> [m get 1]
"42"

Perl-style $1, $2, $~, $` etc. are not supported, only explicit MatchData. $~ support in Ruby is already a big ugly hack (in Perl too, but in Perl everything is a big ugly hack, so it doesn't stick out that much). I'd rather see some clever macros for that. If macro solution won't be possible, I'll try adding $~ and friends to RLisp somehow.

And while we're talking about parsers, RLisp has half-decent backtraces (it had them with the old parser too, I just never got to write about it), with file names, line numbers and function names mostly right:

$ ./rlisp.rb
rlisp> (defun f (x) (g x))
#<Proc:0xb7c276dc@STDIN:1>
rlisp> (map f (list 1 2 3))
No such global variable: g
./rlisp.rb:376:in `default_globals'
STDIN:1:in `call'
STDIN:1:in `default'
STDIN:1:in `[]'
STDIN:1:in `f'
stdlib.rl:112:in `map'
stdlib.rl:112:in `send'
stdlib.rl:112:in `map'
STDIN:2:in `call'
STDIN:2:in `run'
./rlisp.rb:787:in `run'
./rlisp.rb:798:in `repl'
./rlisp_grammar.rb:30:in `each_expr'
./rlisp.rb:796:in `repl'
./rlisp.rb:884:in `main'
./rlisp.rb:888

The rewrite also reduced size of RLisp package from 1MB to some 30kB - earlier it included ANTLR jars just in case someone wanted to play with the grammar.

RLisp syntax for literals

Evil cat by scooter.john from flickr (CC-NC)Every language nowadays requires rich syntax for literals.

Here's a table for RLisp. The lists may seems a bit redundant, but it's basically nicely formatted test set data.

I don't think there's sufficient support for String and Regexp literal syntax.

ClassRLispRubyAST RepresentationNotes
TrueClasstruetrueGlobal identifier true
#ttrueGlobal identifier true (translated inside parser)Scheme compatibility only.
FalseClassfalsefalseGlobal identifier false
#ffalseGlobal identifier false (translated inside parser)Scheme compatibility only.
NilClassnilnilGlobal identifier nil
Integer (Fixnum and Bignum)00Integer object
-0-0
55
-5-5
1234567890123456789012345678901234567890Bignum
-12345678901234567890-12345678901234567890
0x12340x1234Hex
-0x1234-0x1234
07550755Octal
-0755-0755
0b100110b10011Binary
-0b10011-0b10011
Float0.00.0Float objects
-0.0-0.0
5.05.0
-5.0-5.0
3.143.14
-3.14-3.14
1e61e6
-1e6-1e6
1.234e61.234e6
-1.234e6-1.234e6
1.234e+61.234e+6
-1.234e+6-1.234e+6
1.234e-61.234e-6
-1.234e-6-1.234e-6
Array (like "lists" in other Lisps)()[][]Bare non-empty lists are considered function applications.
(list 1 2 3)[1, 2, 3][:list, 1, 2, 3]
'(1 2 3)[1, 2, 3][:quote, [1, 2, 3]]
`(1 2 3)[1, 2, 3][:quasiquote, [1, 2, 3]]
`(1 ,(+ 2 3) 6)[1, 2+3, 6][:quasiquote, [1, [:unquote, [:+, 2, 3]], 6]]
`(1 2 ,@(list 3 4 5))[1, 2, *[3, 4, 5]][:quasiquote, [1, [:"unquote-splicing", [:list, 3, 4, 5]], 6]]
(list 1 2 . (list 3 4 5))[1, 2, *[3, 4, 5]][:list, 1, 2, :".", [:list, 3, 4, 5]]. and & have special meaning in function applications, a bit like Scheme "."/Ruby "*", and Ruby "&".
Range[Range new 1 25]1..25Plain method call
[Range new 2 25 false]2..25
[Range new 3 25 true]3...25
(.. 4 25)4..25Global function ..
(... 5 25)5...25Global function ...
[6 .. 25]6..25Method Object#.. in rlisp_support.rb
[7 ... 25]7...25Method Object#... in rlisp_support.rb
Symbol'foo:foo[:quote :foo]When symbols are evaluated, ., & and self are magical, lowercase are RLisp variables, and uppercase (possibly with ::s inside) are Ruby global constants (but RLisp Array means Ruby ::Array really). In data mode (when quoted, or passed to macros) all symbols are equal. To create unique symbol use (gensym).
foo-bar:"foo-bar"[:quote :"foo-bar"]Symbols can contain some characters that require special escaping in Ruby, like -.
'@foo:@foo[:quote :@foo]
'foo?!?!:"foo?!?!"[:quote :"foo?!?!"]
Hash(hash){}(hash ...) is a macro which expands to [Hash new] and some Hash#[]= calls.
(hash 1 => 2 3 => 4){1 => 2, 3 => 4}
(hash 'foo => 42 'bar => 96){:foo => 42, :bar => 96}
(hash foo: 42 bar: 96){:foo => 42, :bar => 96}Hashes indexed by constant Symbols are so common that they have special syntax (generic syntax still works).
Time[Time now]Time.nowPlain method call.
[Time at 1178573823]Time.at(1178573823)Seconds since Unix epoch
String"Hello""Hello"String object
"Hello, world!\n""Hello, world!\n"Backslash escapes work, #{...} does not.
(str "2 + 2 = " (+ 2 2))"2 + 2 = #{2 + 2}"(str ...) global function which calls #to_s and joins results.
Regexp(rx "foo")/foo/Calls to global function (rx ...) which calls Regexp.new.
(rx "(?i:foo)")/foo/i or /(?i:foo)/
(rx "\\d+")/\d+/Double-escaping is necessary.

Monday, May 07, 2007

Quake-style console for KDE - yakuake

yakuake screenshot by taw (public domain)I played with harnir's fvwm-crystal (screenshots) for a few days. I'm back to KDE now, but one feature was simply too awesome to miss - the Quake-style konsole.

It turns out KDE supports Quake-style konsole too with yakuake package (sudo apt-get install yakuake). Before that I used to keep multitab konsole on virtual desktop 1 (empty now), but I like yakuake much more. The only thing I changed was key binding from truly horrible F12 to Alt-` (run yakuake, press down arrow in lower right corner, select Change Access Key...).

You can see yakuake on the screenshot. It supports multiple tabs, automatically hides when it loses focus, and is based on KDE konsole, which in my experience tends to be a much better terminal emulator than all xterm-based stuff.

Sunday, May 06, 2007

How to test glue code ?

Let Me Out!!! by Plastic Jesus from flickr (CC-NC-SA)There's something I don't know about software development, and I'd like to know. Everybody believes Test-Driven Development is the right way to build software, and it's very easy to develop libraries this way, and still reasonably easy to develop applications, but how does one even test glue code ?

The software which got me thinking about this is my iPod-last.fm bridge. It extracts song statistics from an iPod and submits them to last.fm.

It's a bit over 200 lines of code, and it had a few bugs:

  • The magic constant to convert time was off by an hour. My iPod had DST settings wrong, and I miscalculated the constant so the result was right for my iPod.
  • It assumed record size was 16-bytes instead of looking at the headers. Older iPods had 12-byte records.
  • It expected upper-case MD5 in response from last.fm. last.fm switched to sending lower-case MD5s later.
  • Documentation didn't state that Ruby version 1.8.3 or newer is required, and it won't work with Ruby 1.8.2.
Four non-trivial bugs in 200 lines of code is a horrible ratio. What should be done to reduce number of bugs ?

The script is very simple glue code, and it glued things on which it was tested perfectly - Ruby 1.8.4, an iPod with 16-byte records and broken DST settings, whichever version of software last.fm used when I coded the script.

I don't see how it would be possible to test such script. Building simple mock iPod or mock last.fm server wouldn't find any of the bugs, and complex mocks would be far more complex than the script itself, and most likely would contain even more bugs.

I didn't think about testing it with Ruby 1.8.2, as from Linux (crontabbed apt-get upgrade) perspective this version was ancient, it requires effort to install multiple minor versions of Ruby on Ubuntu, and I didn't expect any problems there. After discovering a few more minor incompatibilities, I automatically run unit tests for all my Ruby code on 1.8.4, 1.8.5, 1.8.6, and 1.9. I don't think I need to support 1.8.2 or 1.8.3, but I'd like to be able to know what is supported and what isn't.

I somehow don't see code review finding any of these problems. If I was doing code review, I think I'd be unlikely to question 2082848400 time conversion magic, 16-byte record size, upper case assumption in MD5 rensponse format regexp, or to ask if it would work with very old versions of Ruby would.

Not a single one of this bug could be found by static typing, no matter how strict, or by avoiding side effects.

Releasing the code found 3 of 4 bugs, and they were fixed in a few minutes at most each. Nobody cared enough about time being off by 1 hour to report it, and I found it on my own while playing with iPod settings.

I think it's a typical case for glue code. Very simple code works with some complex systems, and makes assumptions about them, some of which may later prove unwarranted. Testing, code review, static typing and other standard recipes for reducing bug counts don't seem to be working. Most likely they would simply greatly increase amount of code and work. Releasing the code in the wild seems to be very effective (but it failed to find the time bug).

Is it really the best solution we know ?

Saturday, May 05, 2007

Disease eradication

Neska posant el perfil bo by Ainhoa P from flickr (CC-BY)It's safe to say that the vast majority of human suffering is caused by actions and inaction of other people.

Probably the most abominable of these actions is forcing people to live in places where they were born, also known in "developing countries". Contrary to the name, most of these countries aren't "developing" much. A quick glance at life expectancy statistics for 1998 and 2006, an arbitrary cutoff of 73.5 years chosen to be the definition of a "developed country", and naive extrapolation, shows that 83 of 190 countries aren't going to get "developed" in the next 25 years.

Developed countries (life expectancy, and increase since 1998):

  • Andorra - 83.51yr (+0.01)
  • Singapore - 81.71yr (+1.61)
  • San Marino - 81.71yr (+0.61)
  • Japan - 81.25yr (+0.55)
  • Sweden - 80.51yr (+0.91)
  • Switzerland - 80.51yr (+0.91)
  • Australia - 80.50yr (+0.70)
  • Iceland - 80.31yr (+0.91)
  • Canada - 80.22yr (+0.82)
  • Italy - 79.81yr (+0.81)
  • France - 79.73yr (+0.93)
  • Monaco - 79.69yr (+0.89)
  • Liechtenstein - 79.68yr (+0.88)
  • Spain - 79.65yr (+0.85)
  • Norway - 79.54yr (+0.84)
  • Israel - 79.46yr (+0.86)
  • Greece - 79.24yr (+0.84)
  • Austria - 79.07yr (+1.37)
  • Malta - 79.01yr (+1.11)
  • Netherlands - 78.96yr (+0.66)
  • Luxembourg - 78.89yr (+1.79)
  • New Zealand - 78.81yr (+1.01)
  • Germany - 78.80yr (+1.40)
  • Belgium - 78.77yr (+0.97)
  • United Kingdom - 78.54yr (+0.84)
  • Finland - 78.50yr (+1.10)
  • Jordan - 78.40yr (+1.00)
  • Bosnia and Herzegovina - 78.00yr (+6.50)
  • United States - 77.85yr (+0.75)
  • Cyprus - 77.82yr (+1.12)
  • Denmark - 77.79yr (+1.29)
  • Ireland - 77.56yr (+0.76)
  • Portugal - 77.53yr (+1.73)
  • Republic of China (Taiwan) - 77.26yr (+0.86)
  • Albania - 77.24yr (+5.64)
  • Cuba - 77.23yr (+1.03)
  • Kuwait - 77.03yr (+2.53)
  • South Korea - 76.85yr (+2.45)
  • Costa Rica - 76.84yr (+1.04)
  • Chile - 76.58yr (+0.88)
  • Libya - 76.50yr (+1.00)
  • Ecuador - 76.21yr (+5.11)
  • Slovenia - 76.14yr (+1.24)
  • Uruguay - 76.13yr (+0.93)
  • Czech Republic - 76.02yr (+1.52)
  • Argentina - 75.91yr (+0.81)
  • Georgia - 75.88yr (+11.38)
  • Saudi Arabia - 75.46yr (+7.66)
  • Panama - 75.25yr (-0.25)
  • United Arab Emirates - 75.24yr (+1.14)
  • Mexico - 75.19yr (+3.69)
  • Paraguay - 74.89yr (+1.19)
  • Tunisia - 74.89yr (+1.19)
  • Brunei - 74.80yr (+1.20)
  • Poland - 74.74yr (+1.54)
  • Serbia and Montenegro - 74.73yr (+2.33)
  • Dominica - 74.65yr (+1.25)
  • Slovakia - 74.50yr (+0.80)
  • Croatia - 74.45yr (+0.75)
  • Venezuela - 74.31yr (+1.21)
  • Bahrain - 74.23yr (+1.23)
  • Lithuania - 73.97yr (+4.87)
  • Macedonia - 73.73yr (-0.07)
  • Qatar - 73.67yr (+1.27)
  • Saint Lucia - 73.61yr (+1.31)
Developing countries (life expectancy, increase since 1998, time to become "developed"):
  • Algeria - 73.00yr (+3.30, 1.21yr)
  • Brazil - 71.69yr (+8.79, 1.65yr)
  • Sri Lanka - 73.17yr (+1.37, 1.93yr)
  • Oman - 73.13yr (+1.33, 2.23yr)
  • Egypt - 71.00yr (+7.70, 2.60yr)
  • Armenia - 71.55yr (+5.15, 3.03yr)
  • Thailand - 71.95yr (+3.35, 3.70yr)
  • Solomon Islands - 72.66yr (+1.36, 4.94yr)
  • Lebanon - 72.63yr (+1.33, 5.23yr)
  • Estonia - 71.77yr (+2.27, 6.10yr)
  • Marshall Islands - 70.01yr (+4.51, 6.19yr)
  • Mauritius - 72.38yr (+1.38, 6.49yr)
  • Turkey - 72.36yr (+1.36, 6.71yr)
  • Malaysia - 72.24yr (+1.44, 7.00yr)
  • Latvia - 71.05yr (+2.65, 7.40yr)
  • Saint Kitts and Nevis - 72.15yr (+1.45, 7.45yr)
  • Ukraine - 69.68yr (+3.68, 8.30yr)
  • Hungary - 72.40yr (+1.00, 8.80yr)
  • Kyrgyzstan - 68.16yr (+4.76, 8.97yr)
  • Antigua and Barbuda - 71.90yr (+1.40, 9.14yr)
  • Seychelles - 71.82yr (+1.42, 9.46yr)
  • Colombia - 71.72yr (+1.42, 10.03yr)
  • Bulgaria - 72.03yr (+1.13, 10.41yr)
  • People's Republic of China - 72.27yr (+0.87, 11.31yr)
  • Romania - 71.35yr (+1.45, 11.86yr)
  • Philippines - 69.91yr (+2.41, 11.92yr)
  • El Salvador - 71.22yr (+1.52, 12.00yr)
  • Guatemala - 69.06yr (+2.86, 12.42yr)
  • Morocco - 70.66yr (+1.56, 14.56yr)
  • Samoa - 70.72yr (+1.52, 14.63yr)
  • Nicaragua - 70.33yr (+1.63, 15.56yr)
  • Cape Verde - 70.45yr (+1.55, 15.74yr)
  • Kazakhstan - 66.55yr (+3.35, 16.60yr)
  • Iraq - 68.70yr (+2.20, 17.45yr)
  • Palau - 70.14yr (+1.54, 17.45yr)
  • Vietnam - 70.61yr (+1.31, 17.65yr)
  • Burma - 60.70yr (+5.80, 17.66yr)
  • Syria - 70.03yr (+1.53, 18.14yr)
  • Fiji - 69.53yr (+1.63, 19.48yr)
  • Tonga - 69.53yr (+1.63, 19.48yr)
  • Indonesia - 69.57yr (+1.57, 20.03yr)
  • Uganda - 51.59yr (+8.69, 20.17yr)
Stagnant countries (life expectancy, increase since 1998, time to become "developed"):
  • North Korea - 71.37yr (+0.67, 25.43yr)
  • Tuvalu - 68.01yr (+1.71, 25.68yr)
  • Federated States of Micronesia - 69.75yr (+1.15, 26.09yr)
  • Rwanda - 46.96yr (+7.66, 27.72yr)
  • Gabon - 55.02yr (+4.92, 30.05yr)
  • Sao Tome and Principe - 66.99yr (+1.69, 30.82yr)
  • Republic of the Congo - 52.26yr (+4.86, 34.96yr)
  • Bolivia - 65.50yr (+1.80, 35.56yr)
  • Papua New Guinea - 64.93yr (+1.83, 37.46yr)
  • India - 64.35yr (+1.85, 39.57yr)
  • Maldives - 64.06yr (+1.86, 40.60yr)
  • Guyana - 65.50yr (+1.50, 42.67yr)
  • Pakistan - 63.00yr (+1.90, 44.21yr)
  • Haiti - 52.92yr (+3.72, 44.26yr)
  • Nauru - 62.73yr (+1.93, 44.64yr)
  • Eritrea - 58.47yr (+2.67, 45.03yr)
  • Burundi - 50.29yr (+4.09, 45.40yr)
  • Vanuatu - 62.49yr (+1.89, 46.60yr)
  • Comoros - 61.96yr (+1.96, 47.10yr)
  • Cambodia - 58.92yr (+2.42, 48.20yr)
  • Yemen - 61.75yr (+1.95, 48.21yr)
  • Bangladesh - 62.08yr (+1.88, 48.60yr)
  • Kiribati - 61.71yr (+1.91, 49.38yr)
  • Guinea - 49.36yr (+3.76, 51.36yr)
  • Belarus - 68.72yr (+0.72, 53.11yr)
  • Ethiopia - 48.83yr (+3.63, 54.37yr)
  • Nepal - 59.80yr (+2.00, 54.80yr)
  • Togo - 57.01yr (+2.31, 57.11yr)
  • Cote d'Ivoire - 48.62yr (+3.42, 58.20yr)
  • Sudan - 58.54yr (+1.94, 61.69yr)
  • Malawi - 41.43yr (+3.83, 66.99yr)
  • Benin - 52.66yr (+2.46, 67.77yr)
  • Madagascar - 56.95yr (+1.95, 67.90yr)
  • Laos - 55.08yr (+1.98, 74.42yr)
  • Bhutan - 54.39yr (+1.99, 76.82yr)
  • Democratic Republic of the Congo - 51.10yr (+2.30, 77.91yr)
  • Mauritania - 52.73yr (+1.93, 86.09yr)
  • Mozambique - 40.32yr (+2.82, 94.13yr)
  • Moldova - 65.18yr (+0.68, 97.88yr)
  • Mali - 48.64yr (+1.94, 102.52yr)
  • Somalia - 48.09yr (+1.89, 107.56yr)
  • Zambia - 39.70yr (+2.50, 108.16yr)
  • Iran - 69.96yr (+0.26, 108.92yr)
  • Niger - 43.50yr (+2.20, 109.09yr)
  • Ghana - 58.47yr (+1.07, 112.37yr)
  • Burkina Faso - 48.45yr (+1.75, 114.51yr)
  • Uzbekistan - 64.19yr (+0.49, 152.00yr)
  • Tajikistan - 64.56yr (+0.46, 155.48yr)
  • Namibia - 43.93yr (+1.43, 165.43yr)
  • Azerbaijan - 63.35yr (+0.45, 180.44yr)
  • Turkmenistan - 61.39yr (+0.49, 197.71yr)
  • The Gambia - 53.75yr (+0.55, 287.27yr)
  • Angola - 38.43yr (+0.13, 2158.15yr)
  • Grenada - 64.53yr (+0.03, 2392.00yr)
  • Zimbabwe - 37.82yr (+0.02, 14272.00yr)
  • Kenya - 47.99yr (-0.01, never)
  • Russia - 67.10yr (-0.10, never)
  • Barbados - 72.59yr (-0.41, never)
  • Peru - 69.53yr (-0.47, never)
  • Honduras - 69.30yr (-0.60, never)
  • Central African Republic - 43.39yr (-0.61, never)
  • Trinidad and Tobago - 66.73yr (-1.27, never)
  • Dominican Republic - 71.44yr (-1.76, never)
  • Jamaica - 73.33yr (-1.87, never)
  • Guinea-Bissau - 46.61yr (-2.39, never)
  • Suriname - 68.96yr (-2.44, never)
  • Belize - 68.44yr (-2.46, never)
  • Mongolia - 64.52yr (-2.78, never)
  • Afghanistan - 42.90yr (-3.00, never)
  • Senegal - 58.90yr (-3.30, never)
  • Chad - 47.18yr (-3.32, never)
  • Equatorial Guinea - 49.70yr (-3.90, never)
  • Cameroon - 50.89yr (-3.91, never)
  • Nigeria - 46.74yr (-4.86, never)
  • Botswana - 33.87yr (-5.43, never)
  • Sierra Leone - 39.87yr (-5.43, never)
  • The Bahamas - 65.54yr (-5.56, never)
  • Tanzania - 45.24yr (-7.06, never)
  • Swaziland - 33.22yr (-7.18, never)
  • Djibouti - 43.10yr (-7.70, never)
  • South Africa - 43.27yr (-7.83, never)
  • Liberia - 38.89yr (-12.11, never)
  • Lesotho - 34.47yr (-16.33, never)
Extrapolation has very high error, so each entry on the list can be disputed, but in any case by forcing people who happen to be born in these countries to live there instead of letting them move somewhere else, people are responsible for making new generations of people suffer.

There's nothing inherent about the yet unborn people in Lesotho and other fucked-up places that makes places where they live so fucked up. They just happen to be born where there is no basic healthcare, no viabale economy, no political institutions guaranteeing rule of law, respect for human rights, democracy and so no, and are forced to suffer from all that.

But xenophobia is as strong as ever. The same people who are obsessed about the Holocaust, WTC attacks, and Virginia Tech shootings, are complacent about other genocides, suicide attacks on universities and other places and other catastrophes, at least as long as they don't affect people of European descent, or whatever definition of "us" and "them" they have.

It seems that humans are evolutionarily highly predisposed to xenophobia, and typically care very little about hurting "out-group" people through actions, and not at all about doing so through inaction.

Disease eradication

Over one billion people suffer from diseases caused to being born in the wrong place. In otherwise healthy individuals, and with access to modern medicine, few infectious diseases are live-threatening or seriously debilitating. So most people living in the developed world tend not to care too much about them.

But infectious diseases have a nice side to them. They can in principle be completely eradicated. If the only way to get sick is by contacting an infectious organism, and the only way these organisms can proliferate is by infecting people, then it's enough to break this cycle once and a disease is completely eliminated.

So far only one major diseas was completely eradicated - smallpox. The cost was about US$300 million, that is about one day of ocupation of Iraq (probably a few days, adjusting for inflation). Before it was eradicated, it killed about 2 million people a year.

Currently the largest eradication effort concerns poliomyelitis, with number of cases dropping from estimated 350,000 a year to less than 2,000 a year, but eradication is taking much longer than planned. The cost so far was US$ 3 billion, or ten Iraq-days.

Efforts to eradicate dracunculiasis reduced number of affected people from 3.5 million in the 1980s to about 10,000 in 2005, at cost of US$ 90 millions, or 7 Iraq-hours.

There are realistic plans to eradicate measles, mumps, and rubella. A limited effort to fight measles reduced number of yearly deaths 871,000 (1999) to 454,000 (2004), at cost of $308 million. that is about 1 Iraq-day.

Not every disease can be easily eradicated. Efforts to eradicate malaria failed when mosquitos which transmitted it became immune to insecticides. It doesn't mean we cannot try a different way in the future. We known a lot more about malaria and about mosquitos than we used to, and we have better methods.

It wouldn't be eradication, but occurrence of many diseases like cholera can be greatly reduced by ensuring access to safe drinking water, better nutrition, and letting people in refuge camps (some of the most dangerous places to live) back home, or wherever, as long as it doesn't involve not thousands of people cramped in small space in unsanitary conditions. The cost would be a few Iraq-days at most.

Conclusion

The conclusion is simple - if occupation of Iraq ceased, and the money was spent on global health care instead (for just a month or two), it would save tens of millions of people a year from death, hundreds of millions a year from other infections, and made the world a much better place. And somehow I just don't see it happening.

Wednesday, May 02, 2007

Kingmaker

”The king of Cats”  Kissie!!! by rubyran from flickr (CC-BY)Long time ago, before Web 2.0, before Web 1.0 even, there was a computer game "Kingmaker", closely based on a board game with the same name. Out of nostalgia I wanted to try it out again, a decade later.

The first impression - abandonware websites are not what they used to be. The download links often don't work, and one of the website was bold enough to require you to comment spam some blog with link to them before it would let you download. One spam per downloaded file (a few MBs). And even then the only way to download is using their download manager program, possibly infested with some spyware.

Using a trivial "workaround" of creating an html file with link to them and deleting it immedately afterwards I got Kingmaker zip, badly scanned manual, and something that claimed to be "a crack", but didn't really do anything.

The game worked fine in a dosbox. To start it typed dosbox -c 'mount c ~/kingmaker', then c:, and finally king. The correct sound settings from the startup menu were "Adlib/Sound Blaster" and "No Speech". I don't really remember if speech worked in the original version, but it doesn't in the dosbox, and it's not really important for the gameplay anyway.

In the default settings dosbox uses very little CPU, and the game is slow. Pressing Ctrl-F12 a few times tells it to use more CPU, Ctrl-F11 to use less, Alt-Enter to toggle fullscreen mode, and Ctrl-F10 to release the mouse - normally once the mouse is captured it stays in the dosbox. That's all I needed to learn about dosbox.

The game

In the game the human player and between 1 to 5 computer players control factions in War of the Roses (1455 - 1487), each trying to get their royal heir crowned, and all other eliminated. The rolay heirs are merely puppets here, the faction can change their mind about who has the most rightful claim as often as they wish.

First, a few gameplay tips.
  • When fighting a battle, it's usually better to control manually than to let computer decide the outcome. That's true for pretty much every game where there is such a choice. In manually controlled battles there are no stalemates, and your nobles won't risk their lives when fighting against much smaller enemy armies. You also have a change of winning with a somewhat smaller force - in computer-controlled battles the only possible outcomes are "bigger force wins", and "stalemate" (plus a small chance of each noble dying). You should only let computer decide when you have no chance anyway, and want a chance of killing some enemies or saving your army by a stalemate.
  • When you control the only king (or Chancellor if there are 2 or no kings), you can summon the Parliament. This can easily win the game - you can give your nobles a lot of titles and offices, and force enemies to come to the Parliament, even if they're hiding in Ireland or Calais. If there are too many titles and offices, give some of them to the least important enemy nobles.
  • The easy win strategy is grabbing any royal heir, keeping them safely somewhere, and hunting small enemy nobles. You will gradually grow in power, while others will keep fighting each other. Just keep harassing others and keep your nobles safe.
  • You can move mercenaries between nobles at any time. Just move them each turn to wherever they're most needed. You can move bishops too - if you have a crownable heir somewhere and need to crown them, just reattach bishops to the right noble. And you can move ships, but that's just to
  • If your nobles are captured in battle, computer will sometimes demand ransom. Usually the ransom isn't high, so just accept it. You can demand ransom too, but it's safer to execute them all.
  • In basic plague mode stay in castles not cities, to avoid plagues. In advanced plague mode move away from infected area.
  • In advanced weather mode if you get "severe storms, movement restricted", you cannot fight, so don't even bother. Computer doesn't seem to know about it.
  • Don't bother conquering random castles and cities. The chance of your nobles dying is much greater than the benefit. Conquer only if there is something inside (heir or enemies), or you have some use for the castle/city.
  • If there are many factions, the resource stack is going to become empty. Lost titles and offices do not get back to the stack (they can only be given by the Parliament), only nobles, bishops, and soldiers do. If the stack is empty, and one noble dies in your turn (killed by you, or by enemy), you will get them. So you can safely take 10-troop noble and attack (computer control) a big enemy army. Either they die and you get them back anyway, or they die and kill someone in process, and you might get that someone. Or you can attack small enemy nobles with your big armies (manual control) - you will get whoever you killed.
The biggest problem with the game are long periods during which nothing happens. After initial chaotic phase all factions consolidate their armies, keep their heirs safe somewhere, and everybody is waiting for some random event to tip the balance. It can take half an hour, and then suddenly something happens and the game is over. AI is not too smart, for one it never summons the Parliament. The interface is a bit annoying, especially for splitting armies and for Parliament, but not much more so than in other old games.

The DRM

The game contains an unusual DRM. You are shown a picture of some location, and have to look it up it the manual and type. The locations are (by page, some pages contain no picture):
  • 5 ALNWICK
  • 6 ARUNDEL
  • 7 ASHBY
  • 8 BAMBURGH
  • 9 BATH
  • 11 BELVOIR
  • 12 BRISTOL
  • 14 CAERNARVON
  • 15 CAISTER
  • 16 CANTERBURY
  • 17 CARDIFF
  • 18 CARISBROOKE
  • 19 CARLISLE
  • 20 CHESTER
  • 21 CHICHESTER
  • 22 CHIRK
  • 24 CONISBOROUGH
  • 25 CONWAY
  • 26 CORFE
  • 28 DOVER
  • 29 FRAMLINGHAM
  • 30 HARLECH
  • 32 HELMSLEY
  • 33 HEREFORD
  • 36 LEEDS
  • 37 LICHFIELD
  • 38 LINCOLN
  • 39 LONDON
  • 40 LUDLOW
  • 41 NORWICH
  • 42 NOTTINGHAM
  • 44 OXFORD
  • 46 RABY
  • 47 RICHMOND
  • 48 ROCKINGHAM
  • 49 SALISBURY
  • 50 ST.ALBANS
  • 52 ST.DAVID'S
  • 54 TATTERSHALL
  • 57 WARWICK
  • 59 WELLS
  • 60 WINDSOR
  • 61 WINGFIELD
  • 62 YORK
The pictures in the scanned manual which is available online are big black blobs, so you can't use them. The only easy way I found was starting up a second instance of Kingmaker and checking all possible answers there (the game gives an 11-page range, so it's not that bad). You have to go through this only once - then simply don't quit the program.

One could also make a lot of screenshots and automatically extract the pictures, but I was too lazy for that.