size, as in our discussion above, which is critical for all mini-batch GD.

4.  Number of hidden layers, which is a hyper-parameter related to model structure.

5.  LR decay, as in our discussion above.

6.  Regularization lambda, which is used to reduce variance and avoid overfitting.

7.  Activation functions, which is used to add nonlinear elements.

8.  Adam *$\beta_1$*, *$\beta_2$* and *$\epsilon$*, as in our discussion above.

We can notice that hyper-parameters included in the list are not only related to training, but also determine the structure of model. , And therefore in the next section, we only talk about the number of hidden layers, regularization lambda, and activation function.\

Model Design-Related Hyper-Parameters
-------------------------------------

The number of hidden layers(*d*) is a critical parameter for determining the overall structure of neural networks, which has a direct influence on the final output [@hinton06]. Deep learning networks with more layers are likely to obtain more complex features and relatively higher accuracy. Upscaling the neural network by adding more layers is a regular method of achieving better results. In this way, a baseline structure is repeated to increase the receptive field. For example, users can choose ResNet-18 to ResNet-200 according to their need for 