It wasn't just a caste marker in India. Muslim invaders from Central Asia, Afghanistan and Iran prided themselves on being superior to the darker Indians. During the Bangladesh Civil War, the lighter skinned west Pakistanis looked down on the darker skinned Bangladeshis (note that Islam was involved, not the Hindu caste system).