this projection setting the “unrelaxed" ADMM iteration turns out to be exactly equivalent to the RRR iteration with time-step $\beta=1$. On the other hand, the most direct way to understand why RRR works at all is to consider $\beta\to 0$ (Figure \[fig:fig1\]) where RRR and ADMM differ. The analysis of this limit and details on the RRR/ADMM relationship are given in appendix \[sec:RRRappendix\].

Organization and notation {#sec:notation}
=========================

This paper is written in the style of a tutorial. Unsupervised training in the loss-free style of learning is provided through a series of examples. The examples are ordered with increasing complexity and correspond to the three types of network shown in Figure \[fig:fig3\]. Readers only interested in deep networks should begin with the first of these, on non-negative matrix factorization (NMF), as many of the same elements are used even in this simplest case. Proofs of mathematical results are placed in the appendix for readability.

![The three types of network featured in this tutorial: (a) single layer network for non-negative matrix factorization, (b) multi-layer classifier network, (c) autoencoder network. Networks are not required to be layered, except for special sets of nodes : data layer (blue), code layer (green), class layer 