for step-activation, are parameterized by $\Delta$. All weights are normalized with norm $\Omega$.

Non-negative matrix factorization {#sec:NMF}
=================================

In non-negative matrix factorization (NMF) one tries to express non-negative data vectors $y_1,y_2,\ldots$ as non-negative mixtures of a set of non-negative feature vectors $w_1,w_2,\ldots, w_n$. If the data vectors have length $m$, then in terms of the $m\times n$ matrix of feature vectors $W$, we seek a representation of the data as $y_1= W x_1$, $y_2=W x_2$, ... where the mixture vectors $x_1, x_2,\ldots$ have length $n$. When there are $|K|$ data, arranging the data vectors and mixture vectors into matrices as well, the $|K|$ representations take the form of the factorization $Y=W X$.

If there exists a factorization where both $W$ and $X$ have no zeros, then it is highly non-unique because $W'=W A$ and $X'=A^{-1}X$ gives another NMF for an arbitrary $n\times n$ matrix $A$ suitably close to the identity. This $n^2$-dimensional family of factorizations collapses to a much smaller one, and NMF becomes considerably harder, when $W$ or $X$ or both have zeros. In general, including this case, a NMF is always non-unique with respect to permuting and scaling the $n$ columns of $W$. We will take advantage of 