We propose a novel quantitative measure to predict the performance of a deep neural network classifier, where the measure is derived exclusively from the graph structure of the network. We expect that this measure is a fundamental first step in developing a method to evaluate new network architectures and reduce the reliance on the computationally expensive trial and error or "brute force" optimisation processes involved in model selection. The measure is derived in the context of multi-layer perceptrons (MLPs), but the definitions are shown to be useful also in the context of deep convolutional neural networks (CNN), where it is able to estimate and compare the relative performance of different types of neural networks, such as VGG, ResNet, and DenseNet. Our measure is also used to study the effects of some important "hidden" hyper-parameters of the DenseNet architecture, such as number of layers, growth rate and the dimension of 1x1 convolutions in DenseNet-BC. Ultimately, our measure facilitates the optimisation of the DenseNet design, which shows improved results compared to the baseline.
 Deep neural networks (DNN) have achieved outstanding results in several classification tasks (Huang et al., 2017; He et al., 2016) . There is some theoretical understanding of the workings of individual elements, such as convolutional filters, activation funtions, and normalisation (Goodfellow et al., 2016; LeCun et al., 2015; Schmidhuber, 2015) . However, current ideas behind the DNN graph design are still based on ad-hoc principles (Mishkin et al., 2017) . These principles are largely qualitative and tend to improve classification accuracy -examples of these principles include: an increase of the network depth (Szegedy et al., 2015) , and an increase of the representation dimensionality (by, for example, expanding the number of channels in deeper parts of the DNN) (Huang et al., 2017; He et al., 2016) . We notice that an effective DNN graph design is largely independent of the data set, as long as the type of data (e.g., images) and task (e.g., classification) are similar. Hence, we argue that good design principles can be encoded in a quantitative measure of the graph and should be justified by a quantitative assessment of the DNN architecture performance. An alternative way of designing a DNN graph structure is based on (meta-)optimisation methods (Jenatton et al., 2017; Kandasamy et al., 2019; Mendoza et al., 2016; Snoek et al., 2012) . Although useful in practice, such optimisation methods add little to our understanding of the design principles of new DNN graphs and are computationally challenging to execute. DNNs form a hierarchical structure of filters that can be seen as a directed graph. The first layers of this graph contain neurons that are active for low level patterns, such as edges and patches (in the case of images) (Zeiler & Fergus, 2014) , while deeper layer neurons are active for more complex visual patterns, such as faces or cars, formed by a hierarchical combination of a large number of simpler filters (Zeiler & Fergus, 2014) . In such representation, each neuron behaves like a binary classifier of the visual pattern learned by the neuron. Also, the strength of the activation is related to how well the pattern is matched. We argue that this linear separability promoted by the neurons is the key ingredient behind an effective quantitative measure of model performance. In this paper, we introduce a new measure that can be a proxy for DNN model performance. This proposed measure is first formulated in the context of Multi Layer Perceptrons (MLPs) to quantitatively predict the classification accuracy of the model. Then, we extend the applicability of the measure to predict the classification accuracy of the following CNNs: VGG (Simonyan & Zisserman, 2014) , ResNet (He et al., 2016) and DenseNet (Huang et al., 2017) . The experiments demonstrate how this quantity can be used to predict the "correct" depth of a simple feed forward DNN with constraints on the parameter budget. The experiments also show how the proposed quantity can be used to improve the design of DenseNet (Huang et al., 2017) and in the study of the effects of some important "hidden" hyper-parameters such as the dimension of 1 × 1 convolutions in the bottlenecks of DenseNet-BC, the number of layers, and the growth rate. Our measure (Z) is calculated based only on the graph structure of the model and assumes that the initialisation and training strategies are close to optimal. We argue that the choice of these strategies is important because it guarantees the realisation of the model potential for a particular classification problem. However, we see these strategies as somewhat orthogonal to network design. We show in Fig. 3 that the value of Z is a good predictor of the optimal depth Λ for a simple feed forward CNN. We also show that the value of Z is a good predictor of accuracy for small values of r (channels of bottleneck layer), and for larger values of r, while Z "saturates", the accuracy tends to remain stable. We conjecture that this happens because regularisation techniques are likely to reduce the effective number of channels when this allows an increase in the number of paths. We find indicative support of this mechanism from the study of the dimensionality of the PCA decomposition of MLP classifiers ( Fig. 2(right) ). Figure 6: This example shows the rules to calculate the contribution to the total number of paths given by each layer in a NN. Each N i can be ≤ than the corresponding channels in the NN. We will leave the proof for this conjecture for a future work, note that this behaviour does not undermine the method -the implementation of models with optimal Z appears to offer best performance with the minimum number of parameters. We now provide guidelines on how to formulate Z -see Fig. 6 . When there is a skip connection around a layer, the number of channels in such layer can contribute fully to the paths (N i ). When two layers are in sequence, the contribution to the paths is the difference of the channels after the reduction due to regularisation (N i − N i−1 ). When the filter size changes (e.g., 1 × 1 followed by 3 × 3 convolution), the channels of the first layer need to be divided by the ratio of the dimension of the filters (
A quantitative measure to predict the performances of deep neural network models. The paper proposes a novel quantity that counts the number of path in the neural network which is predictive of the performance of neural networks with the same number of parameters. The paper presents a method for counting paths in deep neural networks that arguably can be used to measure the performance of the network.