acyclic graph the nodes can be labeled such that $i<j$ for every edge $i\to j$ and inference is well defined by evaluating for $j$ increasing sequentially. In our constraint-based training even the acyclic property can be relaxed. All of our applications feature layered networks; networks with simple cycles appear in the autoencoder example.

Upper case symbols denote sets, such as the geometrical constraint sets $A$ and $B$ of RRR. We use $K$ for the set of data items in a batch. $E$ is always the set of edges, $D$ the set of *data* nodes, and $C$ is the set of *class* nodes in a classifier or *code* nodes in an autoencoder. The cardinality of discrete sets is indicated by vertical bars, as in the number of edges $|E|$, and $\|\cdots\|$ is always the Euclidean norm.

Greek symbols are reserved for hyperparameters. RRR has a time-step parameter $\beta$ and the parameter $\gamma$ introduced in the appendix. While the flow limit $\beta\to 0$ is the easiest to understand, in most of our experiments we use the Douglas-Rachford value $\beta=1$. The effect of changing $\gamma$ from the standard choice $\gamma=1$ has not been explored. Margins, both at the output of a classifier and 