AbstractBackgroundRecently, target auto‐segmentation techniques based on deep learning (DL) have shown promising results. However, inaccurate target delineation will directly affect the treatment planning dose distribution and the effect of subsequent radiotherapy work. Evaluation based on geometric metrics alone may not be sufficient for target delineation accuracy assessment. The purpose of this paper is to validate the performance of automatic segmentation with dosimetric metrics and try to construct new evaluation geometric metrics to comprehensively understand the dose‐response relationship from the perspective of clinical application.Materials and MethodsA DL‐based target segmentation model was developed by using 186 manual delineation modified radical mastectomy breast cancer cases. The resulting DL model were used to generate alternative target contours in a new set of 48 patients. The Auto‐plan was reoptimized to ensure the same optimized parameters as the reference Manual‐plan. To assess the dosimetric impact of target auto‐segmentation, not only common geometric metrics but also new spatial parameters with distance and relative volume (RV) to target were used. Correlations were performed using Spearman's correlation between segmentation evaluation metrics and dosimetric changes.ResultsOnly strong (|R2| > 0.6,p< 0.01) or moderate (|R2| > 0.4,p< 0.01) Pearson correlation was established between the traditional geometric metric and three dosimetric evaluation indices to target (conformity index, homogeneity index, and mean dose). For organs at risk (OARs), inferior or no significant relationship was found between geometric parameters and dosimetric differences. Furthermore, we found that OARs dose distribution was affected by boundary error of target segmentation instead of distance andRVto target.ConclusionsCurrent geometric metrics could reflect a certain degree of dose effect of target variation. To find target contour variations that do lead to OARs dosimetry changes, clinically oriented metrics that more accurately reflect how segmentation quality affects dosimetry should be constructed.