|
    Tong Yu yutong01@inspur.com\
    Department of AI and HPC\
    Inspur Electronic Information Industry Co., Ltd\
    1036 Langchao Rd, Jinan, Shandong, China Hong Zhu zhuhongbj@inspur.com\
    Department of AI and HPC\
    Inspur (Beijing) Electronic Information Industry Co., Ltd\
    2F, Block C, 2 Xinxi Rd., Shangdi, Haidian Dist, Beijing, China
bibliography:
- 'HPOref.bib'
title: 'Hyper-Parameter Optimization: A Review of Algorithms and Applications'
---

Hyper-parameter, auto-tuning, deep neural network

Introduction
============

In the past several years, neural network techniques have become ubiquitous and influential in both research and commercial applications. In the past 10 years, neural networks have shown impressive results in image classification [@szegedy16; @he16], objective detection [@girshick15; @redmon16], natural language understanding [@hochreiter97; @vaswani2017attention], and industrial control systems [@abbeel16; @hammond17].However, neural networks are efficient in applications but inefficient when obtaining a model. It is thought to be a “brute force” method because a network is initialized with a random status and trained to an accurate model with an extremely large dataset. Moreover, researchers must dedicate their efforts to carefully coping with model design, algorithm design, and corresponding hyper-parameter selection, which means that the application of neural networks comes at a great price. “Based on experience” is generally the most widely used method, which 