AbstractPurpose/Objective(s)To describe a log file–based patient‐specific quality assurance (QA) method and develop an in‐house tool for system performance tracking and dose reconstruction in pencil‐beam scanning proton therapy that can be used for pre‐treatment plan review.Materials/MethodsThe software extracts beam‐specific information from the treatment delivery log file and automatically compares the monitor units (MU), lateral position, and size of each spot against the intended values in the treatment plan to identify any discrepancies in the beam delivery. The software has been used to analyze 992 patients, 2004 plans, 4865 fields, and more than 32 million proton spots from 2016 to 2021. The composite doses of 10 craniospinal irradiation (CSI) plans were reconstructed based on the delivered spots and compared with the original plans as an offline plan review method.ResultsOver the course of 6 years, the proton delivery system has proved stable in delivering patient QA fields with proton energies of 69.4–221.3 MeV and an MU range of 0.003–1.473 MU per spot. The planned mean and standard deviation (SD) of the energy and spot MU were 114.4 ± 26.4 MeV and 0.010 ± 0.009 MU, respectively. The mean and SD of the differences in MU and position between the delivered and planned spots were 9.56 × 10−8± 2.0 × 10−4MU and 0.029/−0.007 ± 0.049/0.044 mm on the X/Y‐axis for random differences and 0.005/0.125 ± 0.189/0.175 mm on the X/Y‐axis for systematic differences. The mean and SD of the difference between the commissioning and delivered spot sizes were 0.086/0.089 ± 0.131/0.166 mm on the X/Y‐axis.ConclusionA tool has been developed to extract crucial information about the performance of the proton delivery and monitor system and provide a dose reconstruction based on delivered spots for quality improvement. Each patient's plan was verified before treatment to ensure accurate and safe delivery within the delivery tolerance of the machine.