BackgroundNeoadjuvant chemotherapy (NAC) has become the standard therapeutic option for early high-risk and locally advanced breast cancer. However, response rates to NAC vary between patients, causing delays in treatment and affecting the prognosis for patients who do not sensitive to NAC.Materials and methodsIn total, 211 breast cancer patients who completed NAC (training set: 155, validation set: 56) were retrospectively enrolled. we developed a deep learning radiopathomics model(DLRPM) by Support Vector Machine (SVM) method based on clinicopathological features, radiomics features, and pathomics features. Furthermore, we comprehensively validated the DLRPM and compared it with three single-scale signatures.ResultsDLRPM had favourable performance for the prediction of pathological complete response (pCR) in the training set (AUC 0.933[95% CI 0.895–0.971]), and in the validation set (AUC 0.927 [95% CI 0.858–0.996]). In the validation set, DLRPM also significantly outperformed the radiomics signature (AUC 0.821[0.700–0.942]), pathomics signature (AUC 0.766[0.629–0.903]), and deep learning pathomics signature (AUC 0.804[0.683–0.925]) (allp< 0.05). The calibration curves and decision curve analysis also indicated the clinical effectiveness of the DLRPM.ConclusionsDLRPM can help clinicians accurately predict the efficacy of NAC before treatment, highlighting the potential of artificial intelligence to improve the personalized treatment of breast cancer patients.Supplementary InformationThe online version contains supplementary material available at 10.1186/s12885-023-10817-2.