Prey animals typically try to avoid being detected and/or advertise to would-be predators that they should be avoided. Both anti-predator strategies primarily rely on colour to succeed, but the specific patterning used is also important. While the role of patterning in camouflage is relatively clear, the design features of aposematic patterns are less well understood. Here, we use a comparative approach to investigate how pattern use varies across a phylogeny of 268 species of cryptic and aposematic butterfly larvae, which also vary in social behaviour. We find that longitudinal stripes are used more frequently by cryptic larvae, and that patterns putatively linked to crypsis are more likely to be used by solitary larvae. By contrast, aposematic larvae are more likely to use horizontal bands and spots, but we find no differences in the use of individual pattern elements between solitary and gregarious aposematic species. However, solitary aposematic larvae are more likely to display multiple pattern elements, whereas those with no pattern are more likely to be gregarious. Our study advances our understanding of how pattern variation, coloration and social behaviour covary across lepidopteran larvae, and highlights new questions about how patterning affects larval detectability and predator responses to aposematic prey.