Cancer immunotherapy is shifting paradigms in cancer care. T cells are an indispensable component of an effective antitumor immunity and durable clinical responses. However, the complexity of the tumor microenvironment (TME), which consists of a wide range of cells that exert positive and negative effects on T cell function and survival, makes achieving robust and durable T cell responses difficult. Additionally, tumor biology, structural and architectural features, intratumoral nutrients and soluble factors, and metabolism impact the quality of the T cell response. We discuss the factors and interactions that modulate T cell function and survive in the TME that affect the overall quality of the antitumor immune response.