We developed a new method to isolate small extracellular vesicles (sEVs) from male and female wild-type and 5xFAD mouse brains to investigate the sex-specific functions of sEVs in Alzheimer’s disease (AD). A mass spectrometric analysis revealed that sEVs contained proteins critical for EV formation and Aβ. ExoView analysis showed that female mice contained more GFAP and Aβ-labeled sEVs, suggesting that a larger proportion of sEVs from the female brain is derived from astrocytes and/or more likely to bind to Aβ. Moreover, sEVs from female brains had more acid sphingomyelinase (ASM) and ceramide, an enzyme and its sphingolipid product important for EV formation and Aβ binding to EVs, respectively. We confirmed the function of ASM in EV formation and Aβ binding using co-labeling and proximity ligation assays, showing that ASM inhibitors prevented complex formation between Aβ and ceramide in primary cultured astrocytes. Finally, our study demonstrated that sEVs from female 5xFAD mice were more neurotoxic than those from males, as determined by impaired mitochondrial function (Seahorse assays) and LDH cytotoxicity assays. Our study suggests that sex-specific sEVs are functionally distinct markers for AD and that ASM is a potential target for AD therapy.