accuracy calculation after the final model is trained. Deep neural networks (DNNs) are a class of ML algorithm that learn nonlinear functions by emulating the computational structure of a mammalian brain. It consists of simple computational units called [*neurons*]{} which are connected with [*synapses*]{}.

The values of the synapses, called [*weights*]{} are learned through the [*back-propagation*]{} algorithm. It iteratively updates the weights of the DNN to find a local minimum of an objective/ cost function. With this algorithm, each sample is an input to the [*feed-forward*]{} step. The output is a [*predicted value*]{} which is compared to a [*label*]{}. The difference between the label and predicted value is used to calculate the [*gradients*]{} which are applied to update the weights. This difference is called the [*cost*]{} and the objective of training is to minimize this value on the training set while ensuring that the value on the validation set decreases as well.

Back-propagation is a special case of [*gradient descent*]{}. Any gradient descent variant uses the update rule $$\begin{aligned}
    \label{graddesc}
    \mathbf{w}'&=&\mathbf{w}+\lambda \nabla_{\mathbf{w}}C\\
    \mathbf{b}'&=&\mathbf{b}+\lambda \nabla_{\mathbf{b}}C.\end{aligned}$$ where $\mathbf{w}$ are the weights, $\mathbf{b}$ the biases, $\lambda$ the learning rate, and $C$ is a cost function to be optimized.

We use the notation of 