Immunotherapies have revolutionized cancer treatment modalities; however, predicting clinical response accurately and reliably remains challenging. Neoantigen load is considered as a fundamental genetic determinant of therapeutic response. However, only a few predicted neoantigens are highly immunogenic, with little focus on intratumor heterogeneity (ITH) in the neoantigen landscape and its link with different features in the tumor microenvironment. To address this issue, we comprehensively characterized neoantigens arising from nonsynonymous mutations and gene fusions in lung cancer and melanoma. We developed a composite NEO2IS to characterize interplays between cancer and CD8+ T-cell populations. NEO2IS improved prediction accuracy of patient responses to immune-checkpoint blockades (ICBs). We found that TCR repertoire diversity was consistent with the neoantigen heterogeneity under evolutionary selections. Our defined neoantigen ITH score (NEOITHS) reflected infiltration degree of CD8+ T lymphocytes with different differentiation states and manifested the impact of negative selection pressure on CD8+ T-cell lineage heterogeneity or tumor ecosystem plasticity. We classified tumors into distinct immune subtypes and examined how neoantigen-T cells interactions affected disease progression and treatment response. Overall, our integrated framework helps profile neoantigen patterns that elicit T-cell immunoreactivity, enhance the understanding of evolving tumor-immune interplays and improve prediction of ICBs efficacy.