With the miniaturization and high-frequency requirements of quartz crystal sensors, microscopic issues affecting operating performance, e.g., the surface roughness, are receiving more and more attention. In this study, the activity dip caused by surface roughness is revealed, with the physical mechanism clearly demonstrated. Firstly, the surface roughness is considered as a Gaussian distribution, and the mode coupling properties of an AT-cut quartz crystal plate are systematically investigated under different temperature environments with the aid of two-dimensional thermal field equations. The resonant frequency, frequency–temperature curves, and mode shapes of the quartz crystal plate are obtained through the partial differential equation (PDE) module of COMSOL Multiphysics software for free vibration analysis. For forced vibration analysis, the admittance response and phase response curves of quartz crystal plate are calculated via the piezoelectric module. The results from both free and forced vibration analyses demonstrate that surface roughness reduces the resonant frequency of quartz crystal plate. Additionally, mode coupling is more likely to occur in a crystal plate with a surface roughness, leading to activity dip when temperature varies, which decreases the stability of quartz crystal sensors and should be avoided in device fabrication.