Most people were simply angry that the prices were higher, but the Joi group was angry about the trading with foreigners itself, they even suggested excluding foreigners as a rule.
世人は物価の高騰を怒ったのであるが、攘夷派は攘夷と唱えながら外夷と通商していること自体を怒ったのである。