Objective Perceived cognitive dysfunction in people with fibromyalgia (FM), “fibrofog,” is commonly reported and has been demonstrated in neurocognitive testing. Distractibility and inattention have been implicated as potential contributors to fibrofog, but the role of environmental distractions has not been explored. In this study, ambulatory assessment methods were used to examine whether FM is related to more environmental distractions and to examine the impact of distractions on subjective and objective cognitive functioning. Methods Fifty people with FM and 50 age‐, sex‐, and education‐matched controls without FM completed 8 consecutive days of ambulatory assessments. Five times per day, participants reported perceived cognitive functioning and environmental distractions and completed validated tests of processing speed and working memory. Results The FM group reported distractions in a higher proportion of the ambulatory cognitive testing sessions (40.5%) compared with the group without FM (29.8%; P < 0.001) and more often reported multiple simultaneous distractions. For both groups, sound was the most common distraction. The group with FM reported more distractions caused by light, and the group without FM reported more social distractions. Group differences in subjective and objective cognitive functioning were not augmented during distraction relative to during periods of no distraction. There were no group differences in within‐person changes in cognitive functioning as a function of distraction. Conclusion The group with FM reported more distractions than the group without FM; both groups reported poorer processing speed when distracted, and the effects of distraction on test performance did not differ significantly by group. Findings suggest that sensitivity to environmental distractions may play a role in the experience of cognitive dysfunction in FM.