To address the poor characteristics of low strength and poor toughness in phosphogypsum-based construction material, this study investigates the influence of different diameters, lengths, and dosages of polyvinyl alcohol (abbreviated as PVA) fibers on the workability and mechanical properties of phosphogypsum-based construction material. The results show that as the length and dosage of PVA fibers increase, the flowability of the slurry gradually decreases, and the setting time also shortens. With an increase in the diameter of PVA fibers, the rate of decrease in flowability slows down, and the rate of shortening of setting time also gradually slows down. Moreover, the inclusion of PVA fibers significantly improves the mechanical strength of the specimens. When PVA fibers with a diameter of 15 μm, length of 12 mm, and dosage of 1.6% are used, the phosphogypsum-based construction material reinforced with PVA fibers exhibits optimal performance. Under this mixing ratio, the strength values of the specimens for flexural strength, bending strength, compressive strength, and tensile strength are 10.07 MPa, 10.73 MPa, 13.25 MPa, and 2.89 MPa, respectively. Compared to the control group, the strength enhancements are 273.00%, 164.29%, 15.32%, and 99.31%, respectively. SEM scanning of the microstructure provides a preliminary explanation for the mechanism of how PVA fibers affect the workability and mechanical properties of phosphogypsum-based construction material. The findings of this study can provide a reference for the research and application of fiber-reinforced phosphogypsum-based construction material.