the training results more reproducible [@bergstra2013making].\
In recent years, HPO has become increasingly necessary because of two rising trends in the development of deep learning models. The first trend is the upscaling of neural networks for enhanced accuracy [@tan19b].Empirical studies have indicated that in most cases, more complex machine learning models with deeper and wider layers work better than do those with simple structures [@he16; @zagoruyko16; @huang19].The second trend is to design a tricky lightweight model to provide satisfying accuracy with fewer weights and parameters [@ma18; @sandler18; @tan19a]. In this case, it is more difficult to adapt empirical values because of the stricter choices of hyper-parameters. Hyper-parameter tuning plays an essential role in both cases: a model with a complex structure indicates more hyper-parameters to tune, and a model with a carefully designed structure implies that every hyper-parameter must be tuned to a strict range to reproduce the accuracy. For a widely used model, tuning its hyper-parameters by hand is possible because “the ability to tune by hand” depends on experience and researchers can always borrow knowledge from previous works. This is similar for models at a small scale. However, for a larger model or newly published models, the wide 