If democracy is based on majority ruling a minority, then by that logic a majority-white population has the right to control minority populations. This isn't about "limiting freedoms", it's about repressing something people find distasteful.