Rubber concrete (RC) exhibits high durability due to the rubber admixture. It is widely used in a large number of fatigue-resistant structures. Mesoscale studies are used to study the composition of polymers, but there is no method for fatigue simulation of RC. Therefore, this paper presents a finite element modeling approach to study the fatigue problem of RC on the mesoscale, which includes the random generation of the main components of the RC mesoscale structure. We also model the interfacial transition zone (ITZ) of aggregate mortar and the ITZ of rubber mortar. This paper combines the theory of concrete damage to plastic with the method of zero-thickness cohesive elements in the ITZ, and it is a new numerical approach. The results show that the model can simulate reasonably well the random damage pattern after RC beam load damage. The damage occurred in the middle of the beam span and tended to follow the ITZ. The model can predict the fatigue life of RC under various loads.