Alfalfa (Medicago satiuaL.) is a major forage legume in semi-arid regions such as North China Plain and is the material foundation for the development of herbivorous animal husbandry. How to improve the yield of alfalfa per unit area from a technical perspective and achieve high-yield cultivation of alfalfa is the focus of research by scientific researchers and producers. To evaluate the effects of irrigation and P fertilization as well as the P residual effect on alfalfa yield, we conducted a six-year (2008–2013) field experiment in loamy sand soil. There were four irrigation levels (W0: 0 mm, W1: 25 mm, W2: 50 mm, W3: 75 mm per time, four times a year) and three P fertilization levels (F0: 0 kg P2O5ha−1, F1: 52.5 kg P2O5ha−1, F2: 105 kg P2O5ha−1per time, twice a year). The highest dry matter yield (DMY) was obtained in the W2F2 treatment, with an annual mean of 13,961.1 kg ha−1. During 2009–2013, the DMY of first and second-cut alfalfa increased significantly with increasing irrigation levels, whereas the opposite pattern was observed in fourth-cut alfalfa. Regression analysis revealed that the optimal amount of water supply (sum of seasonal irrigation and rainfall during the growing season) to obtain maximum DMY was between 725 and 755 mm. Increasing P fertilization contributed to significantly higher DMY in each cut of alfalfa during 2010–2013 but not in the first two growing seasons. The mean annual DMY of W0F2, W1F2, W2F2, and W3F2 treatments was 19.7%, 25.6%, 30.7%, and 24.1% higher than that of W0F0 treatment, respectively. When no P fertilizer was applied in F2 plots in 2013, soil availability and total P concentrations, annual alfalfa DMY, and plant nutrient contents did not differ significantly compared with those in fertilized F2 plots. Results of this study suggest that moderate irrigation with lower annual P fertilization is a more environmentally sound management practice while maintaining alfalfa productivity in the semi-arid study area.