Due to the traditional use of manual methods for the parameter adjustment of a nonlinear beta transform, which is inefficient and unstable, an adaptive image enhancement algorithm based on a variable step size fruit fly optimization algorithm and a nonlinear beta transform is proposed. Utilizing the intelligent optimization characteristics of the fruit fly algorithm, we automatically optimize the adjustment parameters of a nonlinear beta transform to achieve better image enhancement effects. Firstly, the dynamic step size mechanism is introduced into the fruit fly optimization algorithm (FOA) to obtain a variable step size fruit fly optimization algorithm (VFOA). Then, with the adjustment parameters of the nonlinear beta transform as the optimization object, and the gray variance of the image as the fitness function, an adaptive image enhancement algorithm (VFOA-Beta) is obtained by combining the improved fruit fly optimization algorithm with the nonlinear beta function. Finally, nine sets of photos were used to test the VFOA-Beta algorithm, while seven other algorithms were used for comparative experiments. The test results show that the VFOA-Beta algorithm can significantly enhance images and achieve better visual effects, which has a certain practical application value.