Hard and brittle materials have excellent physical and mechanical performance, which are widely applied in the fields of microelectronics and optoelectronics. However, deep-hole machining of hard and brittle materials is very difficult and inefficient due to the high hardness and brittleness of these materials. To improve the quality and efficiency of deep-hole machining of hard and brittle materials, according to the brittle crack fracture removal mechanism of hard and brittle materials and the cutting model of the trepanning cutter, an analytical cutting force prediction model of hard and brittle materials processed using a trepanning cutter is established. This experimental study of K9 optical glass machining shows that as the feeding rate increase, the cutting force increase, and as the spindle speed increase, the cutting force decrease. By comparing and verifying the theoretical and experimental values, the average errors of axial force and torque are 5.0% and 6.7%, respectively, and the maximum error is 14.9%. This paper analyzes the reasons for the errors. The results indicate that the cutting force theoretical model can be used to predict the axial force and torque of machining hard and brittle materials under the same conditions, which provides a theory for optimizing machining process parameters.