No, opposing immigration of certain ethnic/religious groups is labelled racist.  The protest in Quebec City was anti-Islam, not just anti-immigration.

People of all political stripes have a problem with how the temporary workers program has expanded beyond it's original purpose of supplying seasonal labour, for example.  That's not racist because it's not based on race.