Data-driven discovery of partial differential equations (PDEs) has recently made tremendous progress, and many canonical PDEs have been discovered successfully for proof of concept. However, determining the most proper PDE without prior references remains challenging in terms of practical applications. In this work, a physics-informed information criterion (PIC) is proposed to measure the parsimony and precision of the discovered PDE synthetically. The proposed PIC achieves satisfactory robustness to highly noisy and sparse data on 7 canonical PDEs from different physical scenes, which confirms its ability to handle difficult situations. The PIC is also employed to discover unrevealed macroscale governing equations from microscopic simulation data in an actual physical scene. The results show that the discovered macroscale PDE is precise and parsimonious and satisfies underlying symmetries, which facilitates understanding and simulation of the physical process. The proposition of the PIC enables practical applications of PDE discovery in discovering unrevealed governing equations in broader physical scenes.