Rolling bearing fault diagnosis is of great significance to the safe and reliable operation of manufacturing equipment. In the actual complex environment, the collected bearing signals usually contain a large amount of noises from the resonances of the environment and other components, resulting in the nonlinear characteristics of the collected data. Existing deep-learning-based solutions for bearing fault diagnosis perform poorly in classification performance under noises. To address the above problems, this paper proposes an improved dilated-convolutional-neural network-based bearing fault diagnosis method in noisy environments named MAB-DrNet. First, a basic model called the dilated residual network (DrNet) was designed based on the residual block to enlarge the model’s perceptual field to better capture the features from bearing fault signals. Then, a max-average block (MAB) module was designed to improve the feature extraction capability of the model. In addition, the global residual block (GRB) module was introduced into MAB-DrNet to further improve the performance of the proposed model, enabling the model to better handle the global information of the input data and improve the classification accuracy of the model in noisy environments. Finally, the proposed method was tested on the CWRU dataset, and the results showed that the proposed method had good noise immunity; the accuracy was 95.57% when adding Gaussian white noises with a signal-to-noise ratio of −6 dB. The proposed method was also compared with existing advanced methods to further prove its high accuracy.