PurposeStudies indicate that adult childhood cancer survivors do not achieve recommended physical activity levels. A deeper understanding of factors that influence their ability to be physically active is essential to identify individuals in need of support. The aim was to explore factors that influence adult childhood cancer survivor’s ability to be physically active.MethodSemi-structured interviews were conducted from June to October 2020 with 20 adult childhood cancer survivors with a median age of 31 (min–max 20–47) years. Interviews were transcribed verbatim and analyzed with qualitative content analysis.ResultsFour main categories: “The impact of environmental factors,” “Personal factors of importance,” “Consequences of the treatment or disease,” and “The impact of support from healthcare” and 10 sub-categories, were identified. Participants described how family habits and encouragement from others influenced their present ability to be physically active. Experienced benefits of physical activity were described as a facilitator for current physical activity while suffering from late complications was identified as a barrier. Participants highlighted the importance of specific and individualized physical activity recommendations.ConclusionThis study includes adult childhood cancer survivors several years after completion of treatment, hence highlighting the importance for support both during treatment and follow-up to sustain their physical activity. Healthcare providers need to identify individuals suffering from late complications, even several years after treatment; provide individualized physical activity recommendations; and educate families and schools about the importance of physical activity in childhood cancer survivorship.Trial registrationThis research project was registered in the Swedish National Database of Research and Development, identifier 273320, December 6, 2019 (https://www.researchweb.org/is/vgr/project/273320)Supplementary InformationThe online version contains supplementary material available at 10.1007/s00520-023-07865-6.