Semaphorin 4D (SEMA4D) is considered a new antitumor target closely related to immune cells. However, understanding the role of SEMA4D in the tumor microenvironment (TME) is limited. In this study, we explored the expression and immune cell infiltration patterns of SEMA4D using multiple bioinformatics datasets and analyzed the relationship between SEMA4D expression with immune checkpoints, tumor mutational load (TMB), microsatellite instability (MSI) and immune function. We detected that SEMA4D is overexpressed in many tumors types, widely enriched in immune cells, and closely associated with TILs, MSI, TMB, as well as T-cell exhaustion-associated immune checkpoints, and thus can broadly affect the immune microenvironment. We further verified the overexpression of SEMA4D in tumor and its distribution in TME by immunohistochemistry, RT-qPCR and flow cytometry, and confirmed that decreased expression of SEMA4D can lead to recovery of T cell exhaustion. In conlusion, this study provides a more comprehensive perspective of SEMA4D regulation of tumor immunity, which provide a new option for cancer immunotherapy.