Microtremor recording tests using an accelerometer were carried out in this paper with the aim of characterizing the effects of seasonally frozen soil on the seismic site response, including the two-direction microtremor spectrum, site predominant frequency, and site amplification factor. The study selected eight typical seasonal permafrost sites in China for site microtremor measurements during both summer and winter seasons. Based on the recorded data, the horizontal and vertical components of the microtremor spectrum, HVSR curves, site predominant frequency, and site amplification factor were calculated. The results showed that seasonally frozen soil increased the predominant frequency of the horizontal component of the microtremor spectrum, while the effect on the vertical component was less noticeable. It indicates that the frozen soil layer has a significant impact on the propagation path and energy dissipation of seismic waves in the horizontal direction. Furthermore, the peak values of the horizontal and vertical components of the microtremor spectrum decreased by 30% and 23%, respectively, due to the presence of seasonally frozen soil. The predominant frequency of the site increased by a maximum of 35% and a minimum of 2.8%, while the amplification factor decreased by a maximum of 38% and a minimum of 11%. Additionally, a relationship between the increased site predominant frequency and the cover thickness was proposed.