The relationship between initial COVID-19 infection and the development of long COVID remains unclear. The purpose of this study was to compare the experience of long COVID in previously hospitalized and non-hospitalized adults in a community-based, cross-sectional telephone survey. Participants included persons with positive COVID-19 test results between 21 March 2021 and 21 October 2021 in Alberta, Canada. The survey included 330 respondents (29.1% response rate), which included 165 previously hospitalized and 165 non-hospitalized individuals. Significantly more previously hospitalized respondents self-reported long COVID symptoms (81 (49.1%)) compared to non-hospitalized respondents (42 (25.5%),p< 0.0001). Most respondents in both groups experienced these symptoms for more than 6 months (hospitalized: 66 (81.5%); non-hospitalized: 25 (59.5),p= 0.06). Hospitalized respondents with long COVID symptoms reported greater limitations on everyday activities from their symptoms compared to non-hospitalized respondents (p< 0.0001) and tended to experience a greater impact on returning to work (unable to return to work—hospitalized: 20 (19.1%); non-hospitalized: 6 (4.5%),p< 0.0001). No significant differences in self-reported long COVID symptoms were found between male and female respondents in both groups (p> 0.05). This study provides novel data to further support that individuals who were hospitalized for COVID-19 appear more likely to experience long COVID symptoms.