In a previous post I have covered how I introduce metaclasses. This post instead goes over some applications of metaclasses I have found interesting.

As metaclasses are a feature of a programming language they are only useful in combination with other features. This idea will drive the division of this text.

Metaclasses and state

Metaclasses can be combined with program’s state to both keep track of derived classes and implement an extensible program. By program state I mean values stored in names at the global level or local level. Let’s see an example problem where metaclasses are useful to achieve both claims.

The problem at hand is that we want a library to work with URLs. You most likely know about HTTP URLs. They all look like the following http://example.com. There are other URLs that you may have seen before: ftp://example.com, file://tmp/example and so on.

All URLs start with a scheme (e.g. http, ftp, file, …) and you may want to have one class in your python program dealing with URLs but many transport classes to retrieve what the URL points to.

The following example shows how to use metaclasses to keep track of instances of a transport class.

import warnings
TRANSPORTS = {}

class TransportMeta(type):
    def __new__(cls, name, bases, class_dict):
        if "scheme" not in class_dict:
            raise ValueError("Missing `scheme` property in transport.")
        if not isinstance(scheme, str):
            raise ValueError("`scheme` property must be a string.")
        transport_scheme = class_dict["scheme"]
        transport_class = super().__new__(cls, name, bases, class_dict)
        if transport_scheme in TRANSPORTS:
            t = TRANSPORTS[transport_scheme]
            warnings.warn("overidding {} scheme transport from {} to {}".format(transport_scheme,t, transport_class))
        TRANSPORTS[transport_scheme] = transport_class
        return transport_class

class Transport(metaclass=TransportMeta):
    scheme = ""

In this example we define a Transport class and a TransportMeta metaclass. Because the metaclass receives the class body as an argument we can both check for presence of a scheme attribute and keep track of which transports are available for a URL’s scheme by storing the mapping from scheme to class in TRANSPORTS.

The second claim we made was that this library is extensible. What we mean by that is that users of the library can add a transport for a scheme without changing the library. This is possible because we have hooked into the class definition system by using metaclasses.

Any class derived from Transport will eventually use TransportMetaas its metaclass and this is true even for classes defined by users of our library.

Metaclasses and reflection

Metaclasses can be combined with reflection to define automatically intermediate representation. Consider the case of a web application where you can keep track of dogs you met.

If you were to use a framework like Django you would be almost done with 2 classes. How is that remotely possible?

class Dog(models.Model):
    good_boy = True
    name = models.TextField()

class DogView(view.ModelView):
    model = Dog

Django make this possible by doing a lot of work for you automatically. Some of this work is menial and does not require metaclasses at all but some can benefit from metaclasses.

Consider the form associated to the DogView: it displays an input field for the dog name but it does not display a boolean field for the good_boy attribute. This is easily implemented by using reflection to distinguish which attributes inherit from models.Field. This approach does not define a DogForm class but still works if the forms api allows you to dynamically construct forms out of their fields.

import inspect

def form_for(m):
    model_form_dict = {}
    model_form_dict.update(dict(inspect.getmembers(m, lambda v: isinstance(v, models.Field))))
    return forms.Form(model_form_dict)

But with metaclasses you can tie the two definitions together. In this example the form is associated to the model by the metaclass

class ModelMeta(type):
    def __new__(cls, name, bases, class_dict):
        form_class_dict = {"model": None}
        for k, v in class_dict.items():
            if isinstance(v, models.Field):
			    form_dict[k] = v
        class_dict["form"] = form_for(form_dict)
        model_class = super().__new__(cls, name, bases, class_dict)
        form.model = model_class
        return model_class

If the API for forms definition instead goes through inheritance from a base class instead the only choice is to dynamically define the form class for the model. In this example we call type two times. The first for creating a Dog_form class derived from forms.Form and the second to create the Dog class derived from models.Model.

Because we cannot return more than one class at a time we link the two classes and only return one.

class ModelMeta(type):
    def __new__(cls, name, bases, class_dict):
        form_class_dict = {"model": None}
        for k, v in class_dict.items():
            if isinstance(v, models.Field):
                form_class_dict[k] = v
        form_class = type("{}_form".format(name), (forms.Form,), form_class_dict)
        class_dict["form"] = form_class
        model_class = super().__new__(cls, name, bases, class_dict)
        form_class.model = model_class
        return model_class

The result is that each model now has a form associated and a ModelView can access it.

Metaclasses and names

Metaclass can introduce names into the local scoping of a class body. Python usualy advocates for explicitness but I think it’s a neat thing to do. In the following example we introduce the names for the day of the week.

days_of_the_week = {
    "Sunday":    0,
    "Monday":    1,
    "Tuesday":   2,
    "Wednesday": 3,
    "Thursday":  4,
    "Friday":    5,
    "Saturday":  6,
}

class Meta(type):
    def __prepare__(cls, *bases, **kwargs):
        return days_of_the_week

class Task(metaclass=Meta):
    pass

class Laundry(Task):
    scheduled_on = Saturday

Metaclasses and ordering

In python all objects have an associated dictionary holding their properties and there is no way to recover the order of declaration of these properties. This is problematic when the order of declaration is important. As an example consider the issue with calling C from python. Passing a value from python to C and viceversa requires you to specify where that value will be in memory, making the relative order of fields in a struct important.

In the following example the fictional API we create is that of a FFIStruct class that gets mapped to a C struct. The FFIMeta class preserves the ordering of the fields by adding a property member_names to each derived class so that this information is preserved.

class member_table(dict):
    def __init__(self):
        self.member_names = []

    def __setitem__(self, key, value):
        if key not in self:
            self.member_names.append(key)
        dict.__setitem__(self, key, value)
		
class FFIStruct(type):
    def __prepare__(cls, name, bases):
        return member_table()

    def __new__(cls, name, bases, class_dict):
        result = type.__new__(cls, name, bases, dict(class_dict))
        result.membed_names = class_dict.member_names
        return result
		
class Spam(FFIStruct):
    opcode = UINT(16)
	...

For more details see PEP 3115 fromw which I have adapted the previous example.