Background: Having a maximum number of people vaccinated was the objective to control the COVID-19 pandemic. We report in this manuscript the factors associated with the willingness to be vaccinated against COVID-19 during the pandemic period. Methods: From April to May 2022, a community-based cross-sectional survey was performed. Participants were randomly selected from four districts in Benin (taking into account the COVID-19 prevalence). Mixed-effect logistic regression models were used to identify the variables associated with COVID-19 vaccine acceptance. Results: A total of 2069 participants were included. The proportion of vaccine acceptance was 43.3%. A total of 24.2% were vaccinated and showed proof of vaccination. The population’s request for vaccination was higher after the third epidemic wave. The district of residence, the education level, a fear of being infected, the channel of information, poor medical conditions, a good knowledge of the transmission mode and symptoms, and good behaviors were significantly associated with vaccine acceptance. Conclusion: The overall acceptance of the COVID-19 vaccine in the Beninese population was relatively high. However, vaccine campaigns in areas with a low acceptance as well as the disclosure of information, particularly on our knowledge of the disease and the safety, side effects, and effectiveness of the COVID-19 vaccines, should be strengthened with adapted and consistent messages.