BackgroundCytoplasmic male sterility (CMS) is the basis of heterosis exploitation. CMS has been used to hybrid production in cotton, but its molecular mechanism remains unclear. CMS is associated with advanced or delayed tapetal programmed cell death (PCD), and reactive oxygen species (ROS) may mediate this process. In this study, we obtained Jin A and Yamian A, two CMS lines with different cytoplasmic sources.ResultsCompared with maintainer Jin B, Jin A anthers showed advanced tapetal PCD with DNA fragmentation, producing excessive ROS which accumulated around the cell membrane, intercellular space and mitochondrial membrane. The activities of peroxidase (POD) and catalase (CAT) enzymes which can scavenge ROS were significantly decreased. However, Yamian A tapetal PCD was delayed with lower ROS content, and the activities of superoxide dismutase (SOD) and POD were higher than its maintainer. These differences in ROS scavenging enzyme activities may be caused by isoenzyme gene expressions. In addition, we found the excess ROS generated in Jin A mitochondria and ROS overflow from complex III might be the source in parallel with the reduction of ATP content.ConclusionROS accumulation or abrogation were mainly caused by the joint action of ROS generation and scavenging enzyme activities transformation, which led to the abnormal progression of tapetal PCD, affected the development of microspores, and eventually contributed to male sterility. In Jin A, tapetal PCD in advance might be caused by mitochondrial ROS overproduction, accompanied by energy deficiency. The above studies will provide new insights into the cotton CMS and guide the follow-up research ideas.Supplementary InformationThe online version contains supplementary material available at 10.1186/s12870-023-04317-5.