Fast and accurate fault diagnosis is crucial to transformer safety and cost-effectiveness. Recently, vibration analysis for transformer fault diagnosis is attracting increasing attention due to its ease of implementation and low cost, while the complex operating environment and loads of transformers also pose challenges. This study proposed a novel deep-learning-enabled method for fault diagnosis of dry-type transformers using vibration signals. An experimental setup is designed to simulate different faults and collect the corresponding vibration signals. To find out the fault information hidden in the vibration signals, the continuous wavelet transform (CWT) is applied for feature extraction, which can convert vibration signals to red-green-blue (RGB) images with the time–frequency relationship. Then, an improved convolutional neural network (CNN) model is proposed to complete the image recognition task of transformer fault diagnosis. Finally, the proposed CNN model is trained and tested with the collected data, and its optimal structure and hyperparameters are determined. The results show that the proposed intelligent diagnosis method achieves an overall accuracy of 99.95%, which is superior to other compared machine learning methods.