published in nature stated that DL allows computational models composed of multiple processing layers to learn data representations with multiple levels of the abstraction. Due to its strong representation learning ability, DL is well-suited to data analysis. Therefore, in the field of intelligent fault diagnosis, many researchers have applied DL-based techniques, such as multi-layer perception (MLP), auto-encoder (AE), convolutional neural network (CNN), deep belief network (DBN), and recurrent neural network (RNN) to various fields. A large number of DL-based intelligent diagnosis algorithms have been proposed in recent years and their classification accuracies have been tending to 100%. However, when different researchers design DL-based intelligent diagnosis algorithms, they often recommend to use different inputs (like time domain input, frequency domain input, time-frequency domain input, wavelet domain input, slicing image input, etc.) and set different hyper-parameters (like the dimension of the input, the learning rate, the batch size, the network architecture, etc.). In addition, few authors make their codes available for evaluation and comparison, and others are difficult to repeat the results completely and correctly. Therefore, unfair comparisons and ineffective improvement may exist in this field. Considering that this field lacks open source codes and benchmark study, it is crucial to evaluate 