Using "Neanderthal" to refer to someone you despise is archaic and somewhat misleading these days.  As a people or species of extinct human, the Neanderthals possessed brains on average larger than our own, and arguably had better eyesight than us as well.