Everyone came to a place as conquerors. The Anglo Saxons came to Britain as conquerors. That didn't mean they got pushed out when the Normans took over.

The ancient Muslim communities that I am discussing were the ones in Spain, Portugal and Italy. I am excluding the Ottoman Empire's territories in the Balkans and Eastern Europe (e.g., the Crimea).  To be fair, the last large scale massacre in Europe was a holdover from the Ottoman times (the massacre in Srebrenica).