You work for me, Computer.

By Brandon Bloom

Structural and Relational Data

Before reading this post, see the introductory post for this series: Particles and Waves.

Among our goals is the desire to understand the duality of data and code. This duo represents one of the most fundamental pairs of particles and waves in software. However, careful selection of a bounded context can reveal particles and waves at any level. Combined with the observation that particles and space are generally easier to recognize and reason about than waves and time, let’s zoom in on data before tackling code.

Data Structures

Perhaps surprisingly, there is only a small number of foundational abstract data types. Wikipedia lists some common examples: List, Set, Multiset (Bag), Map, Graph, Stack, Queue, Priority Queue, etc. If you squint just right, there are really only two fundamental types of data containers: lists and bags. Every simple data structure can be classified as ordered or unordered. Common modes of use, such as FILO stacks and FIFO queues, are really just specializations of more generalized lists. Similar for maps and graphs; these are specialized bags. More complex data structures can be recognized as a careful composite of ordered and unordered substructures. Even the ubiquitous map can be viewed as a bag of lists, where each list is a key/value pair, constraining keys to be unique.

Individual data structures are the traditional particles we think of when we think about the data in our code. From a typical vantage point, the context that bounds these particles is so far away that we generally don’t ever perceive it. It’s only when we take a step back from a fragment of code and look at a complete system that we can perceive a common and relevant bounded context: a “database”.


Numbers, such as 5, or strings, such as "foo", have sufficient structure to be recognized in our shared global context made up of decimal numbers and unicode. However, 5 is useless on its own. We need to know how to interpret it. For that, we need more shared context. Let’s introduce a database with a fruit table and say that our favorite number is a fruit ID. Now we can lookup up the fruit ID and find out that we’re talking about bananas. It turns out that our number particle is just one of many particles in a banana wave traveling through our database medium.

By changing our perspective, we can study other bounded contexts, revealing more waves. When this happens, there tends to be an inversion: ordered becomes unordered and vice versa. Additional inversions are explored in sections below. For now, let’s choose the perspective of a programming language runtime implementer. From this vantage point, the heap is a database. A simple ordered data structure such as a pair, is actually represented with pointers, no different than our banana ID. The pair structure arrises from the interplay between a ordered structure with two pointers and an unordered structure, a map, encoded as memory itself, that translates addresses to objects. Shift your perspective again, from application user-space to operating system kernel-space and the question of whether memory is ordered or unordered depends on your perspective on virtual memory.

One common form of data warrants a special note: graphs. The easiest representation of a graph in a traditional programming language is to utilize memory pointers. However, this representation is impoverished. It forcibly equates nodes with their graphs that they are part of and sets an implicit boundary equal to all of memory. As much as possible, graphs and nodes should be clearly delimited. A graph should rightfully be considered as a database of nodes.


For the sake of concision, I will refer to data built up from other data in the implicit “global” context as “structural data”, or simply “structures”. Similarly, data derived from the relationship between structures and an explicit context will be referred to as “relational data”, or simply “relations”. These shorthands are not to be confused with the more specific concepts of C-style “structures” or traditional relational database “relations”, despite my abuse of the intentional analogies.

A key distinction and advantage of relational data is its inherit openness. One fruit ID may appear in many tables. Adding a new table to a database extends the meaning of that fruit ID without impacting any existing relations. We can also extend the meaning of that one fruit ID by moving the context boundary: switch databases, connect to a second database, talk to an external fruit service, etc. By contrast, structural data is closed. Structural data can only be “extend” by replacing it with a slightly varied structure; appending an element to a four element list produces a new list of five elements.

Time is the most important dimension of openness. Growth occurs over time as we add tables to a database, mmap additional pages in to virtual memory, or simply add keys to a map. The impact of time on the interpretation of data is a massive topic, to be discussed at another — ahem — time.

Cutting Along The Grain

Much pain in software designs stems from a conflict between the programmer’s intuition and the nature of the data they are working with. Each kind of data “wants” to be used in a particular way. Given the ability to recognize structures and relations, we can choose the most appropriate end of this duality spectrum; then choose strategies for working with such data according to its wants.

Here’s a summary of each kind of data’s respective usual wants:

Structures Relations
Order Ordered Unordered
Substructure Positional Associative
Mutability Immutable Mutable
Hierarchy Contain Children Point To Parents
Construction Bottom-Up Top-Down
Cycles Acyclic Cyclic
Limits Finite Infinite

Each of these wants can be overridden by moving the bounds of your context. Stuff tuples in to the context of a table to exploit associativity; or scope the context of a database to a transaction to build relations out of temporary ids, bottom-up.

Again, time is too vast a topic to discuss in detail here. Suffice to say, time can be bounded by “stopping” it and/or by choosing a time slice. Stopped time turns the mutable in to immutable and a time slice can turn the infinite in to the finite. Conversely, stopped or sliced time can be started and opened again: stick an immutable structure in a mutable box or connect a live pipe to a queue.


This perspective has dramatically hastened and improved how I design software. Carefully considering bounded contexts strongly guides many design choices. In situations where I used to find myself flip-flopping on some tradeoff, I now find myself happily choosing and committing to an appropriate blend of structural and relational data. Future posts in this series will revisit this tradeoff through some more concrete examples.

Particles and Waves

If simpler is better, is fewer better? Is one better than two? When designing programming constructs, there is a tendency towards extremist programming. There is power and beauty in a grand unified approach that has just one class of elements. Alluring as singularity may be, these one-element approaches tend to fail in ways that more complex designs do not. Hybrid approaches tend to win in the end. Why? And how should this reality inform our designs? Can we predict and enable hybrid solutions?

During the 17th century, opposing theories of light were being considered: particles vs waves. A great deal of science was conducted on behalf of either camp, but no clear victor emerged. It wasn’t until the 20th century that the dual nature of light was widely accepted. I’m far from the first to notice the parallels to logic and computing. Similarities are plentiful here. We even use the same word, “duality,” to informally describe the frequent occurrence of this two-element, yin-and-yang pattern in computing.

Where do dualities come from? Why are there so many of them? Why aren’t there more trinities? Quaternities?

Binary classifications arrises when we define a domain of discourse, then segregate that domain by a predicate. That is, some elements of the domain possess some property and the other elements do not. When this property is defined in terms of some bounded context, the binary classifications becomes a metaphysical, particle/wave-style duality. An element of such a domain may or may not satisfy such a predicate depending on your frame of reference!

Consider the famous Rubin’s Vase illusion:

In each image, either a vase or a pair of face profiles can be scene. Consider which subset of pixels satisfies the foreground predicate, and which satisfies its negation or dual, the background predicate. While the “meta-image” has a white background, the left and rightmost sub-images have a black background. Inverting the colors can switch which object appears in the foreground. However, adding a border can achieve the same illusion! Despite the center image being a pixel-perfect, color-matching, sub-rectangle of the rightmost image, the vase and faces appear to have swapped between the foreground and background. The border provides the bounded context that governs the foreground/background duality.

Returning to the duality of light: What property separates particles from waves? Unlike waves, particles possess a definite position in space. Waves have indefinite position, existing only with respect to some medium. A physical medium serves as a bounded context. There’s no splash without a pool of water; just as there is no background without a border.

Faced with duality, there is natural tendency to seek a preference between the two sides. For many binary classifications, there is no clear winner. Heads or tails? White or black? There’s no right answer. However, the equation is not as balanced when it comes to context-bounded dualities. Our preference is necessarily situational. On a white canvas, we paint a black vase in the foreground, but just the opposite on a black canvas. We must consider the context.

Frequently, the “particle” side of a duality is easier to recognize and reason about. The particle theory of light was proposed in the 11th century, nearly 600 years prior to the wave theory. What makes waves more difficult to recognize or reason about? Typically, context is implicit. The choice of context is often already made for you. When I set out to write this blog post, the background of my blog was already white. On a small beach, it’s easy to see the particles of sand, but it’s just as easy to see the waves of the ocean. Our relative scale influences our perspective, and that grants dominance to one of the two perspectives on the duality. Luckily, perspective can be shifted. Placing a handful of sand in to a carefully selected context, such as the surface of a speaker, will instantly reveal a wave. Thinking in terms of bounded contexts can put reasoning about waves on a level playing field with particles.

One last bit of metaphysics is worthy of mention here: spacetime. It is especially difficult to reason about time as a contextual bound on space. Notably, many waves only appear when observing a medium over time, or by projecting the time dimension on to a spatial dimension. Despite being unbounded at a universal scale, time is scarce for us mortals. We can’t capture time, holding on to it for study. At best, we can capture information about the past, or make predictions about the future, effectively transforming time in to the space such information takes up.

A good design takes things apart with an eye towards putting them back together again. However, focusing on the pieces to the exclusion of their contexts will yield a poor design. Even if the parts snap together nicely, a gestalt may not arise, or the composition may clash with its surroundings. The particles and waves or spacetime dualities can be found hiding under every rock in computer science: Data vs interfaces, logic with negation, values and state, imperative vs declarative, structural vs relational, and many more. Learning to recognize these dualities will enable you to design better systems and to produce better compositions within existing systems.

In future posts, I intend to discuss a wide range of programming topics with an eye for context and gestalt. We’ll use the lense of duality to deepen our understanding of the design tradeoffs in systems ranging from calculators to databases and beyond. I’ll then abuse the perspective to argue against the fetishization of modern programming ideals, such as functional purity, or the overuse of constructs such as promises and observables.

Versioning Better Than SemVer

I’ve written before about my dislike for SemVer. Recently, Rich Hickey spoke about this very same issue, and more. I’m currently working on some software that I’m hoping people will use, so I decided to make my thoughts on a better versioning scheme clear. Below you’ll find an exerpt from my new project’s documentation. Each point below could be elaborated on in a full post of it’s own, but this summary has value as a checklist of sorts for those discussions.


Software should make explicit what it promises to provide, as well as make explicit the requirements it relies on. Implicit promises or reliance should be avoided. This is a as much a social contract as it is a technical one.


If software relies on an explicit promise, it should never break.


It is often useful to defer commitments. The default social contract should be one of instability. In the absence of explicit promises, everything should be considered unstable, and subject to change.


It should always be possible to provide more. Making additional promises should never cause breakage.


Occasionally, supporting old contracts becomes burdensome. Parties to these contracts should be provided a path to a new contract without breakage.


Retracting promises should always be preceded by sufficient warning in order to avoid breakage. This entails a new contract which specifies deprecated provisions at the same time as providing new promises to migrate to.


Contracts are identified by a ordinal version number and an instant in time. Any contract may accrete new promises or bug fixes by creating an artifact tagged with a new instant. Successive ordinal versions may retract promises deprecated by previous ordinals.


Even in the presence of migrations, grandfathered contracts should be honored.


A contract timeline may be forked by accreting a new contract with a new name.


Platform systems should facilitate openness such that side-by-side systems may interoperate. Where this is not immediately possible, lower layers should be extended to facilitate this before forking higher layers.

Rarely Reversible







Let’s say you have this simple grammar:

A ::= a+

The production A is formed of 1 or more a characters.

An infinite number of substitutions will satisfy the rule.

a -> A
aaaaaa -> A

Here, the direction of the arrow is critically important.

How can you reverse this production rule?

??? <- A


For example, we could parameterize A by a number.

aaaa <-> A(4)


a <- A


AB ::= a | b

a -> AB
b -> AB

??? <- AB


AB ::= a / b

a -> AB
b -> AB

a <- AB





Unsound and Incomplete







There is a class of such tools. They’re called linters.

Slurp & Spit

Does nobody teach fopen anymore?




Why don’t you even consider fopen?

But files aren’t “Web Scale”!




Data Literals: print, read, spit, and slurp


(def db (atom {...}))

(defn save-data []

(defn load-data []


var db = {...};

function saveData() {
  fs.writeFileSync("somefile", JSON.serialize(db));

function loadData() {
  db = JSON.parse(fs.readFileSync("somefile"));






function saveData() {
  fs.writeFileSync("somefile.tmp", JSON.serialize(db));
  fs.renameSync("somefile.tmp", "somefile");


Remember to configure your backups!

Don’t Stop The World



(import '

(def save-agent (agent nil))

(defn save-data []
  (send-off save-agent
    (fn [_]


Code Density









SemVer: A Technical Solution to a Social Problem







Trivial Cycles in Object Oriented Programming



The problem can be summarized in a single line of pseudocode:



Person / Address / Company ball-of-mud


Person / Address / Company untangled




And here it is in Lisp syntax:

(residents (address person))


(-> person address residents)


person |> address |> residents





So here’s my new blog.

