If you look into the Church history, it was very common in many times that there were some cardinals who were radical or even sworn foes of the pope. There were some who even had to leave Rome for security reasons. Very unusual that a pope stripped them from the title (I think to imprison an opposing cardinal or fighting a war against him was more common than that). And some of them later got popes themselves. I think it is historically speaking quite a normality that there are cardinals very strongly opposed to the reigning pope.