Neural network (NN) has been tentatively combined into multi-objective genetic algorithms (MOGAs) to solve the optimization problems in physics. However, the computationally complex physical evaluations and limited computing resources always cause the unsatisfied size of training set, which further results in the combined algorithms handling strict constraints ineffectively. Here, the dynamically used NN-based MOGA (DNMOGA) is proposed for the first time, which includes dynamically redistributing the number of evaluated individuals to different operators and some other improvements. Radio frequency cavity is designed by this algorithm as an example, in which four objectives and an equality constraint (a sort of strict constraint) are considered simultaneously. Comparing with the baseline algorithms, both the number and competitiveness of the final feasible individuals of DNMOGA are considerably improved. In general, DNMOGA is instructive for dealing with the complex situations of strict constraints and preference in multi-objective optimization problems in physics.