methods must do indirectly, and with no guarantees, via terms in the loss function. An example of this arises in non-negative matrix factorization, when the learned non-negative feature vectors are also required to be $s$-sparse. Gradient-based methods would introduce a differentiable, sparsity-promoting regularizer such as the 1-norm on the weights. By contrast, projection to a sparsity constraint is direct and in fact much used in phase retrieval: all negative components of the feature vector are set to zero and the remaining positive components are sorted and all but the $s$ largest are also set to zero.

A more elaborate example of the kind of constraint just described, included in our survey of applications, might arise when we suspect some fraction $p$ of the data in supervised learning has wrong labels. In this case we would choose to train on rather large batches, each having say 1000 items. For each item in the batch the projection to the correct-class-constraint at the output layer would be computed both on the assumption that the label is correct and also for the case that the label is wrong. Both hypotheses have a projection distance, and the distance-increases to the stronger (“label is correct") constraint 