The COVID-19 pandemic has had an adverse impact on the Rohingya and the Bangladeshi host communities, which have been well documented in the literature. However, the specific groups of people rendered most vulnerable and marginalized during the pandemic have not been studied comprehensively. This paper draws on data to identify the most vulnerable groups of people within the Rohingya and the host communities of Cox’s Bazar, Bangladesh, during the COVID-19 pandemic. This study employed a systematic sequential method to identify the most vulnerable groups in the context of Rohingya and Host communities of Cox’s Bazar. We conducted a rapid literature review (n = 14 articles) to list down Most vulnerable groups (MVGs) in the studied contexts during the COVID-19 pandemic and conducted four (04) group sessions with humanitarian providers and relevant stakeholders in a research design workshop to refine the list. We also conducted field visits to both communities and interviewed community people using In-depth interviews (n = 16), Key-informant Interviews (n = 8), and several informal discussions to identify the most vulnerable groups within them and their social drivers of vulnerabilities. Based on the feedback received from the community, we finalized our MVGs criteria. The data collection commenced from November 2020 to March 2021. Informed consent was sought from all participants, and ethical clearance for this study was obtained from the IRB of BRAC JPGSPH. The most vulnerable groups identified in this study were: single female household heads, pregnant and lactating mothers, persons with disability, older adults, and adolescents. Our analysis also found some factors that may determine the different levels of vulnerabilities and risks faced by some groups more than others in the Rohingya and host communities during the pandemic. Some of these factors include economic constraints, gender norms, food security, social safety-security, psychosocial well-being, access to healthcare services, mobility, dependency, and a sudden halt in education. One of the most significant impacts of COVID-19 was the loss of earning sources, especially for the already economically vulnerable; this had far-reaching consequences on individuals’ food security and food consumption. Across the communities, it was found that the economically most affected group was single female household heads. The elderly and pregnant and lactating mothers face challenges seeking health services due to their restricted mobility and dependency on other family members. Persons living with disabilities from both contexts reported feelings of inadequacy in their families, exacerbated during the pandemic. Additionally, the shutdown in the formal education, and informal learning centres in both communities had the most significant impact on the adolescents during the COVID-19 lockdown. This study identifies the most vulnerable groups and their vulnerabilities amid the COVID-19 pandemic in the Rohingya and Host communities of Cox’s Bazar. The reasons behind their vulnerabilities are intersectional and represent deeply embedded patriarchal norms that exist in both communities. The findings are essential for the humanitarian aid agencies and policymakers for evidence-based decision-making and service provisions for addressing the vulnerabilities of the most vulnerable groups.