Insufficient physical activity and unhealthy lifestyle preferences have been significant concerns for decades. This study aimed to determine the perceived barriers to maintaining physical activity among adults in three major cities of Bangladesh and their association with mental health status. This is a cross-sectional study where 400 participants were selected using a multistage sampling technique. Twenty municipal wards were randomly selected from three cities, followed by a convenient selection of the study participants from each ward. Questionnaires about perceived physical activity barriers were developed based on previously published literature. The DASS-21 scale assessed the mental health status of the study participants. Descriptive statistics were applied to narrate the baseline characteristics of the respondents. The Shapiro–Wilk test was used to check the normality of the perceived physical activity scores. Quantile regression analysis was applied to model the physical activity barrier scores depending on several covariates. Five quantiles were used: the 10th, 25th, 50th, 75th, and 90th. A p-value less than 0.05 was considered significant for hypothesis testing. Among the respondents, 68.50% were male, half of them were married, 68.0% belonged to nuclear families, 48.0% completed graduate level education, 34.25% were service holders, one-third of the respondent's working hours were 6–8 h, and 19.50% belongs to the overweight and obese groups. Poor traffic and construction work near the road (60.30%) was seen as the most significant barrier to physical activity. Over half of the respondents stated that lack of time, facilities, and expenses hinder physical activity. Mental health status reported mild to extremely severe levels of depression (32%), anxiety (47%), and stress (42.50%), respectively. Significant associations between the perceived physical activity scores and gender, family type, occupation, income, BMI, anxiety, and depression were obtained. Ensuring a safe environment, facilitating accessibility and availability of low-cost exercise facilities, improving road and traffic conditions, and providing appropriate mental health counseling may help to mitigate physical activity barriers.