Recently I was interested in writing some fancy Makefile but I was not sure what we can do and how we can extend make to make some task more tolerable. How can I reduce the complexity of my Makefile?

Some task can be exported to some external script and then called by make, this can be very useful if you have to process a lot of files.

Consider this, you have a directory with many csv files, let’s say one for every day of the last month, and you want to ask some questions about some aggregate value. Put all your computation in a script and then we can let make do the hard stuff

data/processed/% : data/download/%.csv
  python process.py $< $* > $@

so now using make data/processed/2019-07-01 will do the work, you can also add more rules to chain them and maybe even download the data on the fly.

This is very good when the task can be solved by another programming language easily and especially if we have not yet bought into having make do all the stuff.

Another escape hatch is that you can define ~functions~ text substitution in Makefiles, as an example you could query whether a table is in a database by looking at exit codes for commands

define check_relation
  psql -d $(PG_DB) -c "\d $@" > /dev/null/ 2>&1 ||
endef


arrest :
	$(check_relation) psql -d $(PG_DB) -c \
            "CREATE TABLE $@ (arrest_id INT PRIMARY KEY, \
                              central_booking INT, \
                              arrest_event_id INT, \
                              fbi_code TEXT)"

This is a neat trick stolen from DataMade that is the alternative to having files lying around. Instead of touching a file after having created a table you can just query it.

This is also useful because it shows that make can handle everything that looks like files: everything that make can query with an exit code can be used as input/output.

There are also additional functions available to makefile writers which are listed in the Functions for Transforming Text but they are not for interaction with the system.

The last approach is the chisel one, we are gonna extend make by putting a new function that can be used anywhere in your Makefile.

This requires us to compile our function in a shared loadable module but is very versatile.

The end result will look like this

all:
        @echo The answer to life, the universe and everything is $(answer ) #  notice the space after
        # $(answer) is the value of the variable answer
        # $(answer ) is the value of the function answer

load answer.so

The code for this loadable module can be found here and is pretty straightforward. Our module will be loaded by make using the answer_gmk_setup function, the name of the function is given by the convention {module_name}_gmk_setup and it should register other functions associating them with a name for us to use in the Makefile.

The gmk_add_function will bind the name "answer" to the function make_answer in our Makefile. We can add some validation parameter by having first the minimum number of arguments accepted and then the maximum number of arguments accepted.

The last parameter is used to specify whether or not make should expand the arguments before passing them to our function. What should be expanded? Obviosly the variables!

But what if you cannot or don’t want to write C again? Make is already extensible with Guile so you can write some scheme instead and get the same integration!

The documentation for extending make can be found here