Memory guides behavior across widely varying environments and must therefore be both sufficiently specific and general. A memory too specific will be useless in even a slightly different environment, while an overly general memory may lead to suboptimal choices. Animals successfully learn to both distinguish between very similar stimuli and generalize across cues. Rather than forming memories that strike a balance between specificity and generality,Drosophilacan flexibly categorize a given stimulus into different groups depending on the options available. We asked how this flexibility manifests itself in the well-characterized learning and memory pathways of the fruit fly. We show that flexible categorization in neuronal activity as well as behavior depends on the order and identity of the perceived stimuli. Our results identify the neural correlates of flexible stimulus-categorization in the fruit fly.