Resist the Siren Song of Make

Published: 2022-03-19

Tagged: programming productivity

Make has become a popular tool for executing project-related tasks: running database migrations, validating the local environment, building & pushing docker images, etc. I suspect it's because it makes creating command line interfaces so easy. For example, if you want to streamline building docker images, all you have to write is:

.PHONY: build
build:
    docker build -t ${PROJECT}/${APP} .

Make's magic then gives you a nice CLI like this: make build APP=server

There's no mucking around with arguments, commands, subcommands, or flags. Even if you used a CLI-building library like click or cobra, you'd need 2-5x more code. But there's another awesome thing about Make: it integrates with bash so certain complicated things become simple, like environment validation:

.PHONY: check-docker
check-docker:
    which docker

.PHONY: build
build: check-docker
    docker build -t ${PROJECT}/${APP} .

As in the previous case, coding this up as a "proper CLI" would entail 2-5x the code. And since developers are lazy by nature, makefiles seem like the perfect tool for handling small, repetitive tasks.

However, there's a subtle, costly trap if you go down this path: as your makefiles grow, they quickly become tar pits of complexity and foot-guns. There are a few reason for this.

First, Make boasts its own esoteric language. If you're just starting out, it's easy to ignore this because you're mostly using bash. But as you add more logic, you'll be forced to learn Make's intricacies, like the two-phase execution model or the difference between = and := or target: and target::.

Allow me to illustrate this:

VAR := hello world

.PHONY: validate
validate:
    [ "${VAR}" = "hello world" ]; \
    VAR="good bye world"

.PHONY: run
run: validate
    @echo "running: ${VAR}"

What will print to the terminal if you execute make run?

If you said running: good bye world then you're out of luck--you'll see running: hello world. To correctly update VAR inside validate:, you need to replace VAR="good bye world" with the natural and obvious (/s) $(eval VAR=good bye world).

So by choosing Make, you're also choosing a future where you and your team, including all new members, have to learn Make in addition to whatever language your project is written in. But that's not the only cost.

Makefiles are hard to test. Basically, if you go down this path, you're signing a contract that stipulates "This code will only be tested manually--except when someone forgets." I'm no testing zealot, but having automated tests for complicated, critical code makes me feel good--especially if the language itself doesn't do much to help catch bugs.

As an example, let's take an earlier snippet and introduce a bug:

.PHONY: check-docker
    which docker

.PHONY: build
build: check-docker
    docker build -t ${PROJECT}/${APP} .

Can you see what's wrong? Hint: docker build will run even if you don't have a docker binary in your PATH. Can you see the problem now?

I removed the check-docker target, yet Make continued on to execute build as if nothing's missing. In any other language, I would expect either a run-time or compile-time error on account of calling a function that doesn't exist.

Now, if your makefiles are all of 30 lines long, eyeballs might be good enough to catch bugs like this one. But once they're longer--especially if you begin importing other makefiles--well, good luck catching bugs.

To be clear, I don't think Make is bad. It's probably great when you're working with C or C++, or if you and your team already know Make inside out. But if neither of these are true, using Make will incur costly tech debt that will smack you in the face at the worst moment.

Ok, so if makefiles are no good, what other options do we have?

Bash would be my go-to for small, personal projects. It can execute commands and manage files, which covers almost everything I need. Because of how scoping works, I can even use source to import other bash files and override functions, giving me plenty of flexibility. It even supports unit testing!

But there a few downsides to bash: working with collections is awkward; it invites clever, impossible-to-debug one-liners; it's a source of subtle cross-version and cross-platform problems (this affects makefiles too); and as I've written before, it's horrible at handling non-text input like json or yaml.

Another option is to use a scripting language like Python. It will do everything you need, and because it's a fully-featured language, you can make it as elegant and tested as you need. However, you will have to pay the price of creating the command line interface and managing dependencies. It's probably better suited for complicated projects or mature teams.

My favorite alternative are tools such as rake (Ruby), mage (Go), or invoke (Python). They not only take care of the boilerplate CLI code for you, they also provide simple wrappers for running shell commands, leaving you free to focus on getting shit done.

Here's a stock example from the Invoke docs:

from invoke import task

@task
def clean(c, docs=False, bytecode=False, extra=''):
    patterns = ['build']
    if docs:
        patterns.append('docs/_build')
    if bytecode:
        patterns.append('**/*.pyc')
    if extra:
        patterns.append(extra)
    for pattern in patterns:
        c.run("rm -rf {}".format(pattern))

@task
def build(c, docs=False):
    c.run("python setup.py build")
    if docs:
        c.run("sphinx-build docs docs/_build")

You run it like this: invoke clean build

And here's one from Mage:

//+build mage

package main

import (
    "github.com/magefile/mage/sh"
)

// Runs go mod download and then installs the binary.
func Build() error {
    if err := sh.Run("go", "mod", "download"); err != nil {
        return err
    }
    return sh.Run("go", "install", "./...")
}

You run it like this: mage build

It's more verbose for sure and requires you and your team to install an extra dependency, but for that price you get the full power, sanity, and structure of your favorite programming language.

So unless you have a compelling reason for it, avoid Make's siren song--there are better, simpler tools out there.