are sorted for the 1000 data. The wrong-label hypothesis/projection option would be applied to the $p\times 1000$ data having the greatest distance increases.

Because the two projections at the core of RRR can leverage the power of many standard data analysis algorithms (e.g. SVD for projecting to a low-rank constraint), the scope of RRR in machine learning is potentially very broad. However, in this study our focus is relatively narrow: a scheme for training standard network models based on the particular, neuron-centric constraint . The method is developed through a series of models of increasing sophistication, from non-negative matrix factorization to representation learning. By using a natural warm-start procedure on batches, and varying the batch size, we explore both the on-line case of small batches and the off-line mode where projections are applied potentially to the entire data set. Projections take the place of back-propagation and have a comparable operation count. Each parameter update (iteration) of RRR has about as many operations as one SGD step, scaling as the product of the size of the batch and the number of edges in the network.

To help build the case that constraint-based, loss-free optimization could serve as a low-level computational framework 