RECISTv1.1 (Response Evaluation Criteria In Solid Tumors) is the most commonly used response grading criteria in early oncology trials. In this perspective, we argue that RECISTv1.1 is ambiguous regarding lesion-to-lesion variation that can introduce bias in decision making. We show theoretical examples of how lesion-to-lesion variability causes bias in RECISTv1.1, leading to misclassification of patient response. Next, we review immune checkpoint inhibitor (ICI) clinical trial data and find that lesion-to-lesion heterogeneity is widespread in ICI-treated patients. We illustrate the implications of ignoring lesion-to-lesion heterogeneity in interpreting biomarker data, selecting treatments for patients with progressive disease, and go/no-go decisions in drug development. Further, we propose that Quantitative Systems Pharmacology (QSP) models can aid in developing better metrics of patient response and treatment efficacy by capturing patient responses robustly by considering lesion-to-lesion heterogeneity. Overall, we believe patient response evaluation with an appreciation of lesion-to-lesion heterogeneity can potentially improve decision-making at the early stage of oncology drug development and benefit patient care.