Although he went up to Kyoto in 1863, he resigned as the president of political affairs on March 2nd when Yoshinobu tried to make a compromise with Sonno Joi ha (supporters of the doctrine of restoring the emperor and expelling the barbarians) including the Choshu Domain, which had a strong influence in Kyoto.
翌文久3年（1863年）には上洛するが、京都では長州藩など尊皇攘夷派が強く、慶喜が尊攘派と妥協しようとすると反対して3月2日に総裁職を辞任する。