Unit testing for RLisp

I've been coding RLisp pretty much continuously since about one week before RuPy 2007. Even a temporary switch to different hardware and revision control system did not make me stop.

Turning RLisp into something more widely usable seems like a huge task. Every new language is expected to come with full CPAN out of the box. I don't know if RLisp will ever get there, but I should at least cover the few most important targets, like unit testing and web development frameworks. Of course I' not going to code them on my own. Like every good programmer I'm way too lazy and impatient for that, and and adding nice interfaces to existing Ruby stuff achieves far more in a lot less time and effort.

At first I implemented all tests in plain Ruby with a few helper methods.

class Test_RLisp_stdlib < Test::Unit::TestCase
    def setup
        @rlisp = RLispCompiler.new
        @rlisp.run_file("stdlib.rl")
    end
    def assert_runs(code, expected)
        assert_eql(expected, @rlisp.run(RLispGrammar.new(code).get_expr))
    end
    def test_double
        run_rlisp("(let double (fn (x) (* x 2)))")
        assert_runs("(double 5)", 10)
        assert_runs("(double 11.0)", 22.0)
    end
    ...
end

Testing plain functions worked well enough, testing macros that way turned to be much more difficult, and I wanted to code something in RLisp.

A functional prototype took no time and it looked fairly well:

(ruby-require "test/unit")

(let Test_testing_framework [Class new Test::Unit::TestCase])
(class Test_testing_framework
    (method test_assertions ()
        [self assert_equal 9 (+ 2 7)]
        [self assert_equal 2 (- 6 4)]
        [self assert_in_delta 1.99 2.02 0.05 "Deviation is too high"]
        [self assert_not_equal () nil]
        [self assert_same #f false]
        [self assert_same #t true]
    )
)

In Ruby messages to self have nice syntax, and they're often used for designing DSLs. Unit testing language is a DSL too. Lisps are well known for their DSLs too, except that they build them with macros.

That's how the first RLisp DSL began. Syntactic sugar for test suites (classes) and tests (methods) was straightforward.

(defmacro test (name . body)
   (let test_name [(+ "test_" [name to_s]) to_sym])
   `(method ,test_name () ,@body)
)

(defmacro test-suite (name . body)
  (let class_name [(+ "Test_" [name to_s]) to_sym])
  `(do
    (let ,class_name [Class new Test::Unit::TestCase])
    (class ,class_name
        ,@body
    )
  )
)

It was less obvious what to do with assertions. There are too many assert_* methods, and creating one globally visible macro for each didn't sound like an exactly elegant idea. Making the macros active only within a test suite would be nicer, but I wasn't really convinced it was the best way, and besides I've never done such a thing before, it seemed complicated, and writing nontrivial macros with just defmacro and quasiquotes is very hard.

But I wasn't limited to defmacro and quasiquotes any more - the RLisp pattern matching macro (itself implemented with way too many quasiquotes) was just sitting there waiting for some action.

After some syntactic experiments I came out with this syntax:

(require "rlunit.rl")

(test-suite RLUnit
    (test assertions
         (assert 9 == (+ 2 7))
         (assert 2 == (- 6 4))
         (assert 2 == (- 6 4) msg: "Subtraction should work")
         (assert 96 == (* 8 4 3))
         (assert nil? nil)
         (assert true)
         (assert (> 3 2))
         (assert msg: "assert_block failed" block: (true))
         (assert (not (== 3 7)))
         (assert 1.99 == 2.02 delta: 0.05 msg: "Deviation is too high")
         (assert '() instance_of? Array)
         (assert '(foo bar) instance_of? Array)
         (assert '(foo bar) kind_of? Enumerable)
         (assert () != nil)
         (assert #f same: false)
         (assert #t same: true)
    )
)

I could have converted everything to assert_block, but it seemed more elegant to use specific assertions, mostly to reuse nice failure messages.

(defmacro assert args
    (match args

      ('nil? a)           `[self assert_nil ,a]
      ('nil? a 'msg: ,msg)   `[self assert_nil ,a ,msg]

      (a '=~ b)           `[self assert_match ,b ,a]
      (a '=~ b 'msg: msg) `[self assert_match ,b ,a ,msg]

      (a '!~ b)           `[self assert_no_match ,b ,a]
      (a '!~ b 'msg: msg) `[self assert_no_match ,b ,a ,msg]

      (a '!= b)           `[self assert_not_equal ,a ,b]
      (a '!= b 'msg: msg) `[self assert_not_equal ,a ,b ,msg]

      (a 'same: b)        `[self assert_equal ,a ,b]
      (a 'same: b 'msg: msg)    `[self assert_equal ,a ,b ,msg]

      (a 'not_same: b)    `[self assert_not_same ,a ,b]
      (a 'not_same: b 'msg: msg)    `[self assert_not_same ,a ,b ,msg]

      (a 'kind_of? b)     `[self assert_kind_of  ,b ,a]
      (a 'kind_of? b 'msg msg)   `[self assert_kind_of  ,b ,a ,msg]

      (a 'instance_of? b) `[self assert_instance_of  ,b ,a]
      (a 'instance_of? b ,msg: msg) `[self assert_instance_of  ,b ,a ,msg]

      (a b)               `[self assert_equal ,a ,b]

      (a '== b 'delta: c) `[self assert_in_delta  ,a ,b ,c]
      (a '== b 'delta: c
               'msg: msg) `[self assert_in_delta  ,a ,b ,c ,msg]


      (a '== b)           `[self assert_equal ,a ,b]
      (a '== b 'msg: msg) `[self assert_equal ,a ,b ,msg]

      ('block: (blk))     `[self assert_block & (fn () ,blk)]
      ('block: (blk) 'msg: msg)  `[self assert_block ,msg & (fn () ,blk)]
      ('msg: msg 'block: (blk))  `[self assert_block ,msg & (fn () ,blk)]

      (a)                 `[self assert ,a]
      (a 'msg: msg)       `[self assert ,a ,msg]

     (raise SyntaxError [(cons 'assert args) inspect_lisp])
    )
)

At first it was just a few lines, but as pattern matching is so simple I just kept adding more assertions, and then even Smalltalk-style pretty optional arguments. I'm not convinced wheather raising an exception during macro expansion is a good idea or not. Exceptions in RLisp are much less useful than in Ruby - RLisp compiler compiles RLisp to Ruby and then runs it. Unfotunately backtraces telling you that an exception came from (eval) statement aren't very useful. I pached the compiler so that at least the file names are preserved. It would be far better to have line numbers and function names, but Lisp code is usually heavily processed by macros, so most of the information is already lost before the compilation even begins. I still think it should be possible to get something that works reasonably well most of the time.

The internals have been heavily refactored. For example it's possible to freely mix Ruby and RLisp tests, and RLispCompiler will handle .rlc files and so on on its own. Brad Ediger in his RLisp plugin for Ruby on Rails simply made Ruby require statement handle RLisp files transparently. Doing so would probably require a global instance of RLispCompiler, and I was trying to make it possible to have many independent instances of RLisp running concurrently. Maybe it doesn't make that much sense - after all they will all share a single Ruby environment anyway. Right now running RLisp code takes only a bit more typing than a straight require:

require 'test_parser'
require 'test_rlvm'
require 'test_rlisp'
require 'test_run'
rlisp = RLispCompiler.new
rlisp.run_file 'stdlib.rl'
rlisp.run_file 'test_macros.rl'
rlisp.run_file 'tests/unit_test_test.rl'
rlisp.run_file 'tests/test_rlunit.rl'

$ ./test_all.rb
Loaded suite ./test_all
Started
..............................................................................................
Finished in 13.523565 seconds.

94 tests, 308 assertions, 0 failures, 0 errors

Half of the tests implemented in Ruby, half in RLisp. Here's a sample:

(defmacro assert-macroexpands (source expected-expansion)
    `(assert
        ',expected-expansion
        ==
        (macroexpand-rec ',source)
    )
)

(test-suite OO_Macros
    (test class
        (assert-macroexpands
            (class Foo code code2 code3)
            (send Foo 'instance_eval & (fn ()
                code code2 code3
            ))
        )
        (assert-macroexpands
            (class Foo)
            (send Foo 'instance_eval & (fn ()))
        )
    )
    (test method
        (assert-macroexpands
            (method foo (args) body)
            (send self 'define_method 'foo & (fn (args) body))
        )
    )
    ; Should they autoquote ?
    (test attributes
        (assert-macroexpands
            (attr_reader 'foo)
            (send self 'attr_reader 'foo)
        )
        (assert-macroexpands
            (attr_writer 'foo)
            (send self 'attr_writer 'foo)
        )
        (assert-macroexpands
            (attr_accessor 'foo)
            (send self 'attr_accessor 'foo)
        )
    )
)

There's plenty of other changes. Capitalized names, with ::s or not, are simply treated as Ruby constants. RLisp one-liners are possible with the usual -e syntax, dotted pair notation is supported by many operations including pattern matching (of course as RLisp uses arrays and not linked lists, it wouldn't be possible to support all use cases), #to_s_lisp (used by print) and #inspect_lisp (used by REPL) operations are now separate just like in Ruby. It should simply be a lot more fun to hack.