(1) Background: An increasing amount of research has supported the role of radiomics for predicting pathological complete response (pCR) to neoadjuvant chemoradiation treatment (nCRT) in order to provide better management of locally advanced rectal cancer (LARC) patients. However, the lack of validation from prospective trials has hindered the clinical adoption of such studies. The purpose of this study is to validate a radiomics model for pCR assessment in a prospective trial to provide informative insight into radiomics validation. (2) Methods: This study involved a retrospective cohort of 147 consecutive patients for the development/validation of a radiomics model, and a prospective cohort of 77 patients from two institutions to test its generalization. The model was constructed using T2-weighted, diffusion-weighted, and dynamic contrast-enhanced MRI to understand the associations with pCR. The consistency of physicians’ evaluations and agreement on pathological complete response prediction were also evaluated, with and without the aid of the radiomics model. (3) Results: The radiomics model outperformed both physicians’ visual assessments in the prospective test cohort, with an area under the curve (AUC) of 0.84 (95% confidence interval of 0.70–0.94). With the aid of the radiomics model, a junior physician could achieve comparable performance as a senior oncologist. (4) Conclusion: We have built and validated a radiomics model with pretreatment MRI for pCR prediction of LARC patients undergoing nCRT.