Background: Personality traits, such as neuroticism, that results in vulnerability to stress, and resilience, a measure of stress coping, are closely associated with the onset of depressive symptoms, whereas regular physical activity habits have been shown to reduce depressive symptoms. In this study, the mediating effects of neuroticism and resilience between physical activity duration and depressive symptoms were investigated by a covariance structure analysis. Methods: Between April 2017 and April 2018, 526 adult volunteers were surveyed using self-administered questionnaires. Demographic information, habitual physical activity duration (PAD), neuroticism, and resilience were investigated. The effects of these factors on depressive symptoms were analyzed by a covariance structure analysis. This study was conducted with the approval of the Medical Ethics Committee of Tokyo Medical University. Results: The dose–response curves of physical activity duration and depression scores were U-shaped: the optimal physical activity duration for the lowest depression score was 25.7 h/week. We found that the greater the difference from the optimal PAD, the higher the neuroticism and the lower the resilience, and the more severe the depressive symptoms. Covariance structure analysis demonstrated that neuroticism and resilience significantly and completely mediated the effects of the difference from the optimal PAD on depressive symptoms (coefficient of determinationR2= 0.349). Conclusion: Our study suggests that there is an optimal PAD that reduces depressive symptoms, and that a greater difference from the optimal PAD increases depressive symptoms through neuroticism and resilience.