The incorporation of rubber can enhance concrete’s durability and effectively reduce the damage caused by freeze-thaw cycling (FTC). Still, there has been only limited research on the damage mechanism of RC at the fine view level. To gain insight into the expansion process of uniaxial compression damage cracks in rubber concrete (RC) and summarize the internal temperature field distribution law during FTC, a fine RC thermodynamic model containing mortar, aggregate, rubber, water, and interfacial transition zone (ITZ) is established in this paper, and the cohesive element is selected for the ITZ part. The model can be used to study the mechanical properties of concrete before and after FTC. The validity of the calculation method was verified by comparing the calculated results of the compressive strength of concrete before and after FTC with the experimental results. On this basis, this study analyzed the compressive crack extension and internal temperature distribution of RC at 0, 5, 10, and 15% replacement rates before and after 0, 50, 100, and 150 cycles of FTC. The results showed that the fine-scale numerical simulation method can effectively reflect the mechanical properties of RC before and after FTC, and the computational results verify the applicability of the method to rubber concrete. The model can effectively reflect the uniaxial compression cracking pattern of RC before and after FTC. Incorporating rubber can impede temperature transfer and reduce the compressive strength loss caused by FTC in concrete. The FTC damage to RC can be reduced to a greater extent when the rubber incorporation is 10%.