Python metaclasses patterns
In a previous post I have covered how I introduce metaclasses. This post instead goes over some applications of metaclasses I have found interesting.
As metaclasses are a feature of a programming language they are only useful in combination with other features. This idea will drive the division of this text.
Metaclasses and state
Metaclasses can be combined with program’s state to both keep track of derived classes and implement an extensible program. By program state I mean values stored in names at the global level or local level. Let’s see an example problem where metaclasses are useful to achieve both claims.
The problem at hand is that we want a library to work with URLs. You most likely know
about HTTP URLs. They all look like the following
http://example.com. There are other
URLs that you may have seen before:
file://tmp/example and so on.
All URLs start with a scheme (e.g.
file, …) and you may want to have
one class in your python program dealing with URLs but many transport classes to retrieve
what the URL points to.
The following example shows how to use metaclasses to keep track of instances of a transport class.
In this example we define a
Transport class and a
TransportMeta metaclass. Because the metaclass
receives the class body as an argument we can both check for presence of a scheme attribute and keep track
of which transports are available for a URL’s scheme by storing the mapping from scheme to class
The second claim we made was that this library is extensible. What we mean by that is that users of the library can add a transport for a scheme without changing the library. This is possible because we have hooked into the class definition system by using metaclasses.
Any class derived from
Transport will eventually use
TransportMetaas its metaclass and this is true
even for classes defined by users of our library.
Metaclasses and reflection
Metaclasses can be combined with reflection to define automatically intermediate representation. Consider the case of a web application where you can keep track of dogs you met.
If you were to use a framework like Django you would be almost done with 2 classes. How is that remotely possible?
Django make this possible by doing a lot of work for you automatically. Some of this work is menial and does not require metaclasses at all but some can benefit from metaclasses.
Consider the form associated to the
DogView: it displays an input field for the dog name but it does
not display a boolean field for the
good_boy attribute. This is easily implemented by using
reflection to distinguish which attributes inherit from
models.Field. This approach does not define
DogForm class but still works if the forms api allows you to dynamically construct forms
out of their fields.
But with metaclasses you can tie the two definitions together. In this example the form is associated to the model by the metaclass
If the API for forms definition instead goes through inheritance from a base class instead
the only choice is to dynamically define the form class for the model. In this example
type two times. The first for creating a
Dog_form class derived from
and the second to create the
Dog class derived from
Because we cannot return more than one class at a time we link the two classes and only return one.
The result is that each model now has a form associated and a
ModelView can access it.
Metaclasses and names
Metaclass can introduce names into the local scoping of a class body. Python usualy advocates for explicitness but I think it’s a neat thing to do. In the following example we introduce the names for the day of the week.
Metaclasses and ordering
In python all objects have an associated dictionary holding their properties and there is no way to recover the order of declaration of these properties. This is problematic when the order of declaration is important. As an example consider the issue with calling C from python. Passing a value from python to C and viceversa requires you to specify where that value will be in memory, making the relative order of fields in a struct important.
In the following example the fictional API we create is that of a
FFIStruct class that gets
mapped to a C struct. The
FFIMeta class preserves the ordering of the fields by adding
member_names to each derived class so that this information is preserved.
For more details see PEP 3115 fromw which I have adapted the previous example.