The surface quality of pavement has a significant influence on the driving comfort and the skid resistance performance of roads. The 3D pavement texture measurement provides the basis for engineers to calculate the pavement performance index, such as the international roughness index (IRI), the texture depth (TD), and the rutting depth index (RDI), of different types of pavements. The interference-fringe-based texture measurement is widely used because of its high accuracy and high resolution, by which the 3D texture measurement has excellent accuracy in measuring the texture of workpieces with a diameter of <30 mm. When measuring the engineering products with a larger area (or larger areas), such as pavement surfaces, however, the accuracy is deficient because unequal incident angles due to the beam-divergence angle of the laser beam are ignored during the postprocessing of the measured data. This study aims to improve the accuracy of 3D pavement texture reconstruction based on the interference fringe (3D-PTRIF) by considering the influence of the unequal incident angles during postprocessing. It is found that the improved 3D-PTRIF has better accuracy than the traditional 3D-PTRIF, reducing the reconstruction errors between the measured value and the standard value by74.51%. In addition, it solves the problem of a reconstructed slant surface, which deviates from the horizontal plane of the original surface. Compared to the traditional post-processing method, for the case of smooth surface, the slope can be decreased by69.00%; for the case of coarse surface, the slope can be decreased by15.29%. The results of this study will facilitate accurate quantifying of the pavement performance index by using the interference fringe technique, such as IRI, TD, and RDI.