range of hyper-parameter choices requires a great deal of menial work by researchers, as well as much time and computational resources for trial and error.\
In addition to research, the industrial application of deep learning is a crucial practice in automobiles, manufacturing, and digital assistants. However, even for trained professional researchers, it is still no easy task to explore and implement a favorable model to solve specific problems. Users with less experience have substantial needs for suggested hyper-parameters and ready-to-use HPO tools. Motivated by both academic needs and practical application, automated hyper-parameter tuning services [@golovin17; @amazon18]and toolkits [@liaw18; @msr18]provide a solution to the limitation of manual deep learning designs.\
This study is motivated by the prosperous demand for design and training deep learning network in industry and research. The difficulty in selecting proper parameters for different tasks makes it necessary to summarize existing algorithms and tools. The objective of this research is to conduct a survey on feasible algorithms for HPO, make a comparison on leading tools for HPO tasks, and propose challenges on HPO tasks on deep learning networks. Thus, this remainder of this paper is structured as follows. Section 2 begins with a discussion of key hyper-parameters for 