Understanding sarcasm is a complex ability, which includes several processes. Previous studies demonstrated the possible roles of linguistic and meta-representative factors in understanding sarcasm in school children, while the influence of specific contextual variables still needs to be investigated. Here, we present two studies investigating the possible role of contextual, linguistics, and meta-representative factors in understanding sarcasm in school children. In Study 1, we investigated sarcasm comprehension in 8–9-year-old school children in three different contexts, in which both familiarity and authority were manipulated. We found that understanding sarcasm was facilitated when the conversational partner was characterized by a high level of authority and familiarity (the mother) rather than when the conversational partner was an adult with a lower level of both authority and familiarity (the cashier of a food store). In Study 2, we replicated and extended Study 1 by investigating the possible influence of the same contextual factors but in a more sizeable sample and at different ages: first, third, and fifth grades of primary school. We found that understanding sarcasm improved significantly with age. The results of both studies indicated that understanding sarcasm is influenced by contextual factors. Children at any age better understood sarcasm produced by a speaker with a high level of both familiarity and authority. This ability improved with age. These results expand our understanding of how children infer a speaker’s intentions in sarcasm. This might be particularly of interest to develop possible interventions for children on the Autism Spectrum, who are known to misunderstand sarcasm at different levels of complexity.