Bolting Aspect Oriented Programming on top of Python

The bigest difference between native support and bolting things on top of a programming language is that you can only bolt so much before things start to collapse. In C++ even strings, arrays, and hashtables are bolted on - and while they work just fine any interoperability between different libraries using strings, arrays, and hashtables is almost impossible without massive amount of boilerplate code.

In Perl and Python these basic data structures are native and well supported, but the next step of supporting objects is bolted on. So the objects work reasonably well, but metaprogramming with them is very difficult and limited (in Python) or outright impossible in any sane way (in Perl).

Ruby takes a step further and real object-oriented programming is native, so people can bolt other things on top of it like aspect-oriented programming. AOP in Ruby (before_foo, after_bar, alias_method_chain, mixins, magical mixins, many method_missing hacks etc.) works reasonably well, but I wouldn't want to bolt anything on top of that, or the whole thing would fall apart.

This is the problem with bolting stuff on - bolting stuff on in a valid technique (just like design patterns, code generation and other band-aids), and bolted-on stuff like objects in Perl/Python or arrays/strings/hashtables in C++ do work, they're just infinitely less flexible than native types when it comes to further extending.

But I really miss AOP in Python. Multiple inheritance can kinda simulate very weak kind of mixins, but is rather cumbersome to use. I wanted to write a test suite using aspect-oriented mixins, but there were simply so many super(omg, who).made(up, this, syntax) calls that it looked as painful as Java inner classes. So I thought - would it already collapse if I added a very simple AOP support?

It turned out not to be so bad. Here's a distilled example. There's a bunch of classes inheriting from BaseTest. Their setup methods should be called from superclass down to subclass, while their teardown methods should be called from subclass up to superclass. If there are multiple AOP methods on the same level all should be called, in some consistent order (I do alphanumeric, order of definition would be better but Python metaprogramming isn't powerful enough to do that). It's also possible to override parent's AOP methods (you could even compose AOP method override using super if you really needed). Or you could override the whole setup/teardown method if you really needed - this is very flexible.

class BaseTest(object):
  def setup(self):
    aop_call_down(self, 'setup')
  def teardown(self):
    aop_call_up(self, 'teardown')

class WidgetMixin(object):
  def setup_widget(self):
    print "* Setup widget"
  def teardown_widget(self):
    print "* Teardown widget"

class Foo(BaseTest):
  def setup_foo(self):
    print "* Setup foo"
  def teardown_foo(self):
    print "* Teardown foo"

class Bar(WidgetMixin, Foo):
  def setup_bar(self):
    print "* Setup bar"
  def teardown_bar(self):
    print "* Teardown bar"

class Blah(Bar):
  def setup_blah(self):
    print "* Setup blah1"

  def setup_widget(self):
    print "* Setup widget differently"

  def setup_blah2(self):
    print "* Setup blah2"

  def teardown_blah(self):
    print "* Teardown blah1"

  def teardown_blah2(self):
    print "* Teardown blah2"

The output of a = Bar(); a.setup(); a.teardown() is exactly what we would expect:

* Setup foo
* Setup widget
* Setup bar
* Teardown bar
* Teardown widget
* Teardown foo

The more difficult case of b = Blah(); b.setup(); b.teardown() is also handled correctly - notice that setup of widget mixin was overriden:

* Setup foo
* Setup widget differently
* Setup bar
* Setup blah1
* Setup blah2
* Teardown blah2
* Teardown blah1
* Teardown bar
* Teardown widget
* Teardown foo

The code to call make it possible isn't strikingly beautiful but it's not any worse than some of my Django templatetags.

def aop_call_order(obj, prefix):
  already_called = {}
  for cls in reversed(obj.__class__.mro()):
    for name in sorted(dir(cls)):
      if name[0:len(prefix)+1] != prefix + '_':
        continue
      if not already_called.has_key(name):
        yield(name)
      already_called[name] = True

def aop_call_up(obj, prefix):
  for name in reversed(list(aop_call_order(obj, prefix))):
    getattr(obj, name)()

def aop_call_down(obj, prefix):
  for name in aop_call_order(obj, prefix):
    getattr(obj, name)()

aop_call_order returns a list of methods with names like prefix_* defined in obj's ancestor classes in order of Python's multiple inheritance resolution, falling back to alphabetic if they're on the same layer. Overriding a method in subclass doesn't affect the order, making the "Setup widget differently" trick possible. aop_call_down and aop_call_up methods then call these methods in straight or reverse order.

Of course like all other multilayer bolted-on features, it's going to horribly collapse if you use it together with other metaprogramming feature. If you don't like that - switch to Ruby.

Coming up next - bolting closures on top of Fortran.