Acute myeloid leukemia (AML) is a devastating blood cancer with poor prognosis. Novel effective treatment is an urgent unmet need. Immunotherapy targeting T cell exhaustion by blocking inhibitory pathways, such as PD-1, is promising in cancer treatment. However, results from clinical studies applying PD-1 blockade to AML patients are largely disappointing. AML is highly heterogeneous. Identification of additional immune regulatory pathways and defining predictive biomarkers for treatment response are crucial to optimize the strategy. CD26 is a marker of T cell activation and involved in multiple immune processes. Here, we performed comprehensive phenotypic and functional analyses on the blood samples collected from AML patients and discovered that CD26lowPD-1+CD8 T cells were associated with AML progression. Specifically, the percentage of this cell fraction was significantly higher in patients with newly diagnosed AML compared to that in patients achieved completed remission or healthy controls. Our subsequent studies on CD26lowPD-1+CD8 T cells from AML patients at initial diagnosis demonstrated that this cell population highly expressed inhibitory receptors and displayed impaired cytokine production, indicating an exhaustion status. Importantly, CD26lowPD-1+CD8 T cells carried features of terminal exhaustion, manifested by higher frequency of TEMRAdifferentiation, increased expression of transcription factors that are observed in terminally exhausted T cells, and high level of intracellular expression of granzyme B and perforin. Our findings suggest a prognostic and predictive value of CD26 in AML, providing pivotal information to optimize the immunotherapy for this devastating cancer.