Diabetes mellitus (DM) is a chronic metabolic disorder characterized by hyperglycemia resulting from insulin secretion defects or insulin resistance. The global incidence of DM has been gradually increasing due to improvements in living standards and changes in dietary habits, making it a major non-communicable disease that poses a significant threat to human health and life. The pathogenesis of DM remains incompletely understood till now, and current pharmacotherapeutic interventions are largely inadequate, resulting in relapses and severe adverse reactions. Although DM is not explicitly mentioned in traditional Chinese medicine (TCM) theory and clinical practice, it is often classified as “Xiaoke” due to similarities in etiology, pathogenesis, and symptoms. With its overall regulation, multiple targets, and personalized medication approach, TCM treatment can effectively alleviate the clinical manifestations of DM and prevent or treat its complications. Furthermore, TCM exhibits desirable therapeutic effects with minimal side effects and a favorable safety profile. This paper provides a comprehensive comparison and contrast of Xiaoke and DM by examining the involvement of TCM in their etiology, pathogenesis, treatment guidelines, and other relevant aspects based on classical literature and research reports. The current TCM experimental research on the treatment of DM by lowering blood glucose levels also be generalized. This innovative focus not only illuminates the role of TCM in DM treatment, but also underscores the potential of TCM in DM management.