The best kittens, technology, and video games blog in the world.

Wednesday, June 20, 2007

Object-Oriented dialects of Lisp

Lolcat based on 'Let's see here, Cat Power, Cat Stevens, Purrs... Ooo, Meatloaf!' by Lazy_Lightning from flickr (CC-BY)

Lisp is much older than object oriented programming. When OOP got popular people wanted to do OOP in Lisp too, but it wasn't obvious how to retrofit Lisp to make it object oriented. CLOS for Common Lisp became somewhat popular, but its quite far from what Smalltalkers would call "object oriented", and some people feel it wasn't very Lispy either. Many Lispers like Paul Graham simply gave up on OOP. Others like me decided to code their own object-oriented dialects of Lisp. Solutions they came up with are very different. I checked the following Object-Oriented dialects of Lisp:

  • RLisp - Lisp integrated with Ruby
  • e7 - Lisp integrated with Python
  • goo - Object-Oriented Lisp inspired by Scheme
  • CLOS - Object-Oriented system built on top of Common Lisp
  • I also wanted to check Coke, but it segfaulted at me, and the last thing I felt like doing was debugging broken C code.
In all four I tried to write the same snippet:
  • Define class Point representing 2D points of vectors
  • Define initializer for this class, which takes 2 arguments x and y and returns a Point instance
  • Define a method for stringifying Points, like Ruby's to_s and Python's __str__. If possible, I wanted it to be automatically called by REPL and (print a_point) or its equivalent.
  • Define a method for adding two Points. I wanted to call it + if possible.
  • Create two points, add them, and print the result.
RLisp code:
(let Point [Class new])
(class Point
(attr-reader 'x)
(attr-reader 'y)
(method initialize (x y)
(set! @x x)
(set! @y y))
(method to_s ()
(method + (other)
[Point new (+ [self x] [other x]) (+ [self x] [other y])]))

(let a [Point new 1 5])
(let b [Point new -2 9])
(print [a + b])

e7 code:
(class Point ()
(def (init self x y)
(set-self x x y y)))
(defmethod (+ (self Point) (other Point))
(+ self.x other.x)
(+ self.y other.y)))
(defmethod (print (self Point) f)
(fwrite f (format "<%s,%s>" self.x self.y)))

(def a (Point 1 5))
(def b (Point -2 9))

(print (+ a b))

goo code:
(dc <point> (<any>))
(dp point-x (<point> => <num>))
(dp point-y (<point> => <num>))

(dm point-new (x|<num> y|<num>)
(new point-x x point-y y))

(dm point-add (p1|<point> p2|<point> => <point>)
(+ (point-x p1) (point-x p2))
(+ (point-y p1) (point-y p2))))

(dm write-point (port|<out-port> x|<point>)
(msg port "<%s,%s>" (point-x x) (point-y x)))

(dv a (point-new 1 5))
(dv b (point-new -2 9))

(write-point out (point-add a b))
(newline out)

CLOS code:
(defclass point ()
((x :reader point-x :initarg :x)
(y :reader point-y :initarg :y)))
(defmethod make-point (x y)
(make-instance 'point :x x :y y))
(defmethod point-add (a b)
(+ (point-x a) (point-x b))
(+ (point-y a) (point-y b))))

(defmethod point-print ((p point))
(format t
"<~s,~s>" (point-x p) (point-y p)))

(setf a (make-point 1 5))
(setf b (make-point -2 9))
(point-print (point-add a b))

The first thing I noticed is how different these snippets look in spite of doing pretty much the same thing. The second is that RLisp and e7 are much more concise than goo and CLOS. RLisp and e7 have more syntactic extensions for OO. RLisp supports [ ] syntax for method calls, self for message receiver, and @ivar for instance variables. e7 doesn't go that far and limits itself to obj.attr syntax for attribute access and method calls. In e7 like in Python all attributes are public. In RLisp like in Ruby all attributes are private, and (attr-reader 'x) must be explicitly called.

In RLisp and e7 attributes are dynamic and aren't predeclared anywhere. In goo and CLOS list of attributes is part of class definition. goo even requires attribute types, but will happily accept <any>. Only RLisp seemed to provide a clear way of stringifying objects. recurring-write in goo converts standard objects to strings, but write for some reason ignored extensions of it. In e7 the standard way is overloading print. It seems more limited than stringification method, but it should be possible to print to string buffer instead of a real file. I have no idea what CLOS uses to stringify objects.

The most important ideological difference is treatment of method calls. RLisp makes message sends and function calls separate operations much like Smalltalk-style OOP languages. All others define methods as some kind of generic functions. This means weaker encapsulation and some deep philosophical differences. Coke also separates function calls and message sends, but like I said it was segfaulting, so I was unable to take a closer look at it.

Only in RLisp creating a class and defining stuff inside it are separate operations. Ruby is ambiguous as class Foo can either define a new class or reopen existing class, and RLisp tried to avoid this ambiguity. Of course it's possible to do both with a simple macro. It increased verbosity of code somewhat.

In e7 class is also a constructor. In RLisp it's just an object. CLOS and goo seem to treat classes differently from other objects.

goo and CL provided only REPL environment by default, and didn't like running scripts. RLisp and e7 supported REPL and script mode without any extra hassle.

From a purely subjective point of view, I liked RLisp way most, what's not particularly surprising coming from RLisp's author ;-). Coding in e7 also felt good. On the other hand goo and especially CLOS felt rather unelegant and unpleasant.


Ricky Clarkson said...

Looks to me like CL's defstruct would have suited you better.

Remember, of course, whenever something in Lisp seems overly verbose, you can golf it down by writing a macro.

Anonymous said...

CLOS evolved from earlier object-oriented extensions. Those were LOOPS and Flavors.
Message passing was used earlier. But then it has been changed into generic functions, since that integrates much better into Lisp. Also methods are not included in class definitions and can be updated and extended incrementally. Classes are also objects in CLOS. Nowadays CLOS is embedded in Common Lisp and integrated for example in the type system.

If you want to write MAKE-POINT as a generic function, you can describe the arguments:

(defmethod make-point ((x number) (y number))
  (make-instance 'point :x x :y y))

You can customize how objects are printed in Common Lisp like this:

(defmethod print-object ((p point) stream)
  (print-unreadable-object (p stream :type t :identity t)
    (format stream "~s,~s" (point-x p) (point-y p))()))

Then just call:

CL-USER 4 > (point-add (make-point 1 5) (make-point -2 9))

#<POINT -1,14 200B36B3>

topher said...

Nice comparison!

I liked the e7 code a lot, but was weirded out by the fact that init wasn't defined as a generic method (what does that nesting mean?) and also by the (set-self x x y y))) macro.

I should do some reading, but does that macro require that you've named yourself "self"?

And, of course, RLisp came out looking very good. Here's a question for you, what led you not require instance variable declaration while requiring it for globals and temporaries?

Compatibility with Ruby?

Slava Pestov said...

Too much boilerplate. Let's try in Factor.

- Define a class representing a Point: no point, just use an array of two elements

- Define an initializer: already in the library, 2array

- Define a method for stringifying points: already in the library, unparse

- Define a method for adding two points: already in the library, v+

- Create two points, add them, and print the result:

{ 1 2 } { 3 4 } v+ .

Moral of the story: this example is silly. Certainly not big enough to warrant statements such as "Coding in e7 also felt good. On the other hand goo and especially CLOS felt rather unelegant and unpleasant."

The CLOS, goo and e7 code looks almost identical here.

taw said...

Ricky Clarkson: Other languages have special macros or functions for defining structs too, like (let Point [Struct new 'x 'y]) in RLisp, but I wanted to know how it would look like in the general case.

Anonymous: Common Lisp hasn't exactly been a major success, was it ? I think lack of popularity of not-very-OO CL and huge popularity of OOP back then were related.

topher: init in e7 seems to always be defined like that. Maybe there are technical reasons for it, or maybe it's just stylistic issue.

set-self indeed requires you to call your object self. self in Python and e7 is not special (in Ruby and RLisp it is), it's just a convention:

(defmacro (set-self &rest settings)
`(begin ,@(ZF `(setattr self ',s[0] ,s[1])
for s in (duples settings))))

RLisp mostly follows Ruby object model. Instance variables are stored in a dictionary, and every object can have any instance variables it wants to, no matter which class it belongs to. @foo always refers to attribute @foo of self, and there is no need to declare anything.

Other variables follow normal lexical scoping rules, so some information is needed to distinguish local binding (let) and modifying variable in higher scope (set!).

Slava Pestov: Of course this example is silly, hello world snippets are silly by design, to show what the languages look like without getting into domain issues.

Ricky Clarkson said...

Slava's code could easily be translated to quite readable CL, of course.

(v+ '(1 2) '(3 4)) for example.

He's right that it's a silly example, but it is hard to get a meaningful example down to something readable in a blog format. Is that something to do with the attention span of blog readers?

I don't think the general case is that meaningful, because Lisp programmers tend to get rid of boilerplate.

I don't think the anonymous poster was making any points about CL's popularity. I thought it was reasonably common knowledge that the 'AI winter' killed off CL for a while (but I wasn't there, update me if I'm wrong). Plus, as most OO documentation is written with C++ and related languages in mind, and CL's generic functions differ greatly from those, CLOS probably looks non-OO at a glance from someone who hasn't got the reasoning to fathom how it works.

Anonymous said...

make-point adds nothing to your code. you should instead use make-instance

taw said...

Ricky Clarkson: The OOP-ness issue is much older than C++. The first OOP language was Smalltalk, and it defined OOP as "everything is an object, every action is a message passing".

Then other languages came, implemented something more or less related to Smalltalk OOP, and called it OOP too. Ruby/RLisp, and Javascript are quite close to the Smalltalk model. Python and Java less so. CLOS is so far away from the original OOP that calling it "OOP" is more marketing than accurate technical description. C++ also has very little to do with Smalltalk-style OOP.

There's nothing wrong with disliking Smalltalk-style OOP and preferring something else, but we should be clear what do we mean by "OOP" or everybody will be confused.

Anonymous: make-point is more concise than make-instance, and every other language creates instances this way, so for the sake of fair comparison CLOS version had to implement it too.

Anonymous said...

CLOS is totally fine as an OO extension. I'm using it all the time. There have been extremely large applications (think millions of lines of code) been written using CLOS. I like the design very much. The Art of the Meta-Object Protocol describes its philosophy very well. The book is even described by Alan Kay as one of the best computer science books ever.

Personally I can't find message passing languages with classes better. I'm more interested in Frame languages (on top of CLOS).

Common Lisp is a relative success. CLOS is a success. There are probably ten Common Lisp implementations in active maintenance. Some of them are extremely mature. There are no other Lisp-dialects with comparable quality implementations. Some Scheme implementations aren't bad, though.
If you look around and compare other interactive programming languges with Common Lisp you will find that even though Common Lisp implementations are fully interactive, the code often runs 10 to hundred times faster than say in Ruby, Python or other such languages. That's not that bad. If you look at larger implementations like LispWorks, they implement most of their functionality via CLOS and it does it quite elegantly.

Is CLOS object-oriented or not? There is no single accepted definition for object-oriented. Are prototype-based languages OO? Self? But they have no classes? Is Smalltalk OO? But it does not run methods in parallel (like in Actor languages). Do you really think that a serialized message passing language is 'pure OO'? Do objects act serialized? Really? Actor languages give you message queues for individual objects.

I like also that in CLOS the 'messages' are actually first class objects and have more than one object as a parameter. For me the 'task' is more important than the idea of point to point serialized message sending. Think of the task of printing a document. Who sends the message and who is involved? Do you send a print message to a printer? To a document? To some execution engine? To a cascade of objects sending each other print messages until one does the task somehow? In CLOS you model a PRINT-DOCUMENT task and hand over all objects that are involved.

In the design of an object-oriented extension for a Lisp dialect, it is more important to have an elegant integration than to be 'pure', IMHO. The CLOS model has been widely used in Lisp dialects (Dylan, EuLisp, some Scheme dialects, ISLisp, ...). Seems that lots of people like it. In my book the design of CLOS has been extremely successful. There are even CLOS like extensions for C and other languages. CLOS stands also out, because it is extensible via the Meta-Object Protocol. So CLOS is not single fixed extension, but covers a whole ground of possibilities. You need different types of instances? Persistent instances? Sparse objects? You need different inheritance? You need slot facets? You need multiple-value slots? You need constraints attached to slots? All relatively straight forward. Not everybody may need this power, but Common Lisp was not designed for simple applications, but for the most demanding AI applications (like expert systems, planners, machine translation tools, computer algebra systems, ...) with multiple programming paradigms used at the same time.

taw said...

Anonymous: This "task-oriented" thing you describe is very different than "object-oriented". You seem to like "task-oriented" programming better and that's fine, but as you said objects are not the central thing in CLOS, "tasks" are. And if objects are not the central thing, why do people insist on calling it "object-oriented" ?

And CLOS power is overrated. It cannot even do method_missing (if you think this statement is inaccurate, try coding it) and all the cool functionality that Ruby and Smalltalk build using method_missing. Of course you can design things some other way, maybe a way that wouldn't work in Ruby/Smalltalk. But that's exactly my point - CLOS is significantly different from Smalltalk-style OOP, and it gives you power of completely different kind. Many people prefer Smalltalk-style OOP kind of power.

Anonymous said...

Objects are as central in CLOS as in Smalltalk. Messages in Smalltalk are found in Classes, but not in objects. In CLOS methods are found in generic functions and also not in objects. If you want true objects you need to look at Actor languages. Smalltalk already isn't especially 'more' object-oriented compared to CLOS. The design space of OO is very large. Smalltalk is a serialized class based version of message-passing. A prototype-based version without classes would be more pure. A parallel prototype-based would be even more object-oriented. So, how much sense does it make to say X is more OO than Y, when X isn't pure OO anyway?

The building blocks of CLOS are:

* objects as instances of classes
* classes and meta-classes
* generic functions and methods
* method combinations

and some more...

It is also relatively straight forward to reduce CLOS to a simpler message passing model. But that would be backwards...

method_missing? The comparable thing in CLOS is the generic function NO-APPLICABLE-METHOD. It is called when there is no applicable method and one can write methods for it to handle it.

Anonymous said...

Btw. the first OOP language was not Smalltalk.

Check this out:

Duncan said...

Do you know about Oaklisp and the T system? They are both OO dialects of Scheme.

taw said...

Duncan: I know that there are more OO dialects of Lisp than the ones I listed. Someone on Reddit mentioned Jazzscheme too.

Maybe I'll write a follow-up some day with a few more Lisps.

TechnoMancer said...

Java, C++ and most of those use the simula object model, simula used method "calls", smalltalk used message sends, they still both use object and are therefore object oriented in some way, in common lisp prettymuch all of the CLOS stuff is defined by a set of meta-classes, thus they are all actually objects themselves. There are many languages that allow programming with objects, Io is an interesting one.

Shalin Siriwaradhana said...
This comment has been removed by the author.
Shalin Siriwaradhana said...

Can these object oriented concepts be used for javascripts?


taw said...

Shalin Siriwaradhana: There's ton of OO frameworks for javascript, since few people like the builtin system much. It would be interesting to compare their approaches.