Flat is slow, slow is fast

Published: 2018-04-22

Tagged: architecture learning thoughts

Bob Martin's "Clean Architecture" is a nexus of design thinking for me. It took many loose topics and questions I had and shaped a coherent model of reasoning about software out of them. The relief was short lived, however; While some long-lived gaps in my understanding were filled, new questions and ideas surfaced. One of these is the idea that flat structures are better: more reliable and easier to understand and extend.

The most popular facet of "flat is better" can be seen in "composition over inheritance." In short, this design principle places emphasis on setting up "has-a" relationships between objects instead of "is-a" ones, which leads to shallow inheritance hierarchies. "Has-a" relationships are more flexible, allowing objects to swap out other objects during runtime, which is great of testing. They're also less coupled than "is-a" relationships where every child class is dependent on its parent class, with the deepest children having the most dependencies. These traits make the whole structure easier to understand because each unit can be analyzed and explained in isolation.

Another facet, overlaid on top of the relationship/dependency layer, is the abstraction layer. Building abstractions is how software expresses a problem. At the lowest level, code toggles a current between high and low voltage. But on top of that are built towers of abstraction that tell us whether the code is a web framework or a piece of accounting software. Not enough abstraction, and code is repetitive, verbose, hard to follow and change. Too much though and the code is verbose, hard to navigate and modify. But just the right amount makes code malleable, concise, and elegant. This leads me to paraphrase the words of a certain scientist: "Structures should be as flat as possible, but not flatter!"

Let's zoom out and look at how waterfall and Agile fit in this model. The waterfall process is insanely deep. It starts off from a vague and abstract product, and sprouts branches and subbranches until it you're down to the level of classes, with deadlines, stakeholder meetings, demos, and architecture reviews in between. Visualize it as a tree data-structure, turn it 90 degrees left, add a time axis underneath, and now you have a complex, confusing, software trainwreck. Agile, however, keeps the structure shallow by not planning every single decision, not estimating known and unknown unknowns, and unless it's cargo cult agile, Agile delivers better results (citation needed).

While I can't confirm this with any of the authors of the Agile Manifesto, these ideas aren't new. One branch of history that comes to mind is the US Marine Corps and its MCDP-1 Warfighting (1989) manual, which, while scary sounding, expounds that winning in war is all about increasing your maneuverability. Moving quickly and hitting an unprepared opponent is only the physical manifestation of this, but the core of the matter is aligning the organization to this goal before your opponent can react. As in Agile, one of the recommendations is to push control down towards the people on the ground. This is done through "commander's intent", which means that commanders are to convey their intent, but leave out implementation details, trusting that the people doing the work will know what's best and they'll know it the fastest out of anyone. This flattening of the command structure came as a surprise to me because in my mind the military was the ultimate in top-down command and control. Doesn't this sound like the Agile idea that estimates are no good? That, given enough "client's intent", the best people to invest with authority are the boots-on-the-ground developers?

Let's go back to our humble domain of software development so that I can make one last vague generalization using Rich Hickey's excellent "Simple Made Easy" talk. Hickey argues that simplicity is more valuable than ease. For example, using a framework's magical code generation tool makes it easy to get set up within minutes, but the easiness hides complexity. As time goes by, this incidental complexity will leak more and more, forcing you to spend time fighting it instead of building features. In other words, start flat and only go deep when you consciously decide you have to.

This all leads me to paraphrase yet another saying, flat is slow, slow is fast.