no *activation* function, the hidden layer has *activation* function $f$ and the output layer has *activation* function $g$. It is also possible to have a different *activation* function for each individual neuron.

![A three-layer ANN.[]{data-label="fig:ANN"}](ANN-eps-converted-to.pdf){width="\linewidth"}

The *activation* function in the hidden layer, $f$, is different from the *activation* function in the output layer, $g$. For function approximation, a common choice for the *activation* function for the neurons in the hidden layer is a *sigmoid* or *sigmoid*–like function, while the neurons in the output layer have a *linear* function: $$f(x) = \frac {1} {1 + e^{-ax}},$$ where $a$ is the slope parameter of the *sigmoid* function and $$g(x) = x.$$ The neurons with *non-linear* *activation* functions allow the ANN to learn *non-linear* and *linear* relationships between input and output vectors. Therefore, sufficient neurons should be used in the hidden layer in order to get a good function approximation.

In the example shown in Figure \[fig:ANN\] and with the notations mentioned above, the network propagates the external signal through the layers producing the output signal $z_k$ at neuron $k$ in the output layer

$$\label{eq:output}
\begin{split}
    z_k = g(net_{z_{k}}) & = g(\sum_{j=1}^{m+1} w_{kj} f(net_{y_{j}})) \\
    & = g(\sum_{j=1}^{m+1} w_{kj} f(\sum_{i=1}^{n+1} v_{ji} x_i)).
\end{split}$$

