to take different values for each data item $k$ we impose another consensus constraint: $$\forall\; k,i,j\colon w[k,i\to j]=w_A[i\to j].$$ By consigning these consensus constraints to set $A$ which is inactive when we perform $P_B$ in the RRR algorithm, the neuron-input constraint of set $B$ is able to work with local (otherwise unconstrained) variables.

The third type of constraint implements the activation function $f$ that connects the pre- and post-activation neuron values: $$\label{actcon}
\forall\; k,j\colon f\left(y[k,j]-b[k,j]\right)=x_A[k,j].$$ By consigning this to the $A$ constraints, all the $y$ variables, both in and , are local (within sets $B$ and $A$, respectively). As with the weights we have had to replicate the bias parameters $b$ to make the constraint local to each data item $k$. Consensus is now imposed in set $B$ (as they are allowed to be independent in constraint $A$): $$\forall\; k,j\colon b[k,j]=b_B[j].$$

Depending on the neural network application (classifier, autoencoder, etc.) there may be modifications to and at input-layer, output-layer or code-layer neurons. There may also be constraints for parameter regularization. We describe these in detail, and their membership in $A$ or $B$, in later sections where we study specific applications.

![Step activation function reinterpreted as a constraint set (red).[]{data-label="fig:fig2"}](fig2.eps){width="3.in"}

Now 