Rare variants affecting host defense against pathogens may be involved in COVID-19 severity, but most rare variants are not expected to have a major impact on the course of COVID-19. We hypothesized that the accumulation of weak effects of many rare functional variants throughout the exome may contribute to the overall risk in patients with severe disease. This assumption is consistent with the omnigenic model of the relationship between genetic and phenotypic variation in complex traits, according to which association signals tend to spread across most of the genome through gene regulatory networks from genes outside the major pathways to disease-related genes. We performed whole-exome sequencing and compared the burden of rare variants in 57 patients with severe and 29 patients with mild/moderate COVID-19. At the whole-exome level, we observed an excess of rare, predominantly high-impact (HI) variants in the group with severe COVID-19. Restriction to genes intolerant to HI or damaging missense variants increased enrichment for these classes of variants. Among various sets of genes, an increased signal of rare HI variants was demonstrated predominantly for primary immunodeficiency genes and the entire set of genes associated with immune diseases, as well as for genes associated with respiratory diseases. We advocate taking the ideas of the omnigenic model into account in COVID-19 studies.