In chronic infections and cancers, T lymphocytes (T cells) are exposed to persistent antigen or inflammatory signals. The condition is often associated with a decline in T-cell function: a state called “exhaustion”. T cell exhaustion is a state of T cell dysfunction characterized by increased expression of a series of inhibitory receptors (IRs), decreased effector function, and decreased cytokine secretion, accompanied by transcriptional and epigenetic changes and metabolic defects. The rise of immunotherapy, particularly the use of immune checkpoint inhibitors (ICIs), has dramatically changed the clinical treatment paradigm for patients. However, its low response rate, single target and high immunotoxicity limit its clinical application. The multiple immunomodulatory potential of traditional Chinese medicine (TCM) provides a new direction for improving the treatment of T cell exhaustion. Here, we review recent advances that have provided a clearer molecular understanding of T cell exhaustion, revealing the characteristics and causes of T cell exhaustion in persistent infections and cancers. In addition, this paper summarizes recent advances in improving T cell exhaustion in infectious diseases and cancer with the aim of providing a comprehensive and valuable source of information on TCM as an experimental study and their role in collaboration with ICIs therapy.