If you call someone 'privileged', the underlying message is that they've gotten something because of their status, not because they deserve it. 

So no, 'Guilt' is the desired outcome when you use that word. 

Think about it next time you feel like throwing around the 'P'-word.