What people forget was that until about 1830s when the British banned slavery it was normal throughout the world. 
Romans had slaves (the vast majority white). The Muslim Ottoman Turks had white Christian slaves in their navy and army, the Chinese had slaves, the Africans had slaves, medieval Europe had  slaves (again white).  
You have to condemn everyone in the past if you insist on judging them by todays standards.