After the fourth Ito Cabinet collapsed, he was to assume the position of prime minister, given an imperial command, but he declined and handed it over to Taro KATSURA, seeing that the political situation couldn't offer much prospect (but when he was interior minister of the second Ito Cabinet, he served as provisional prime minister by proxy for about a month, when Ito met a car accident).
第4次伊藤内閣の崩壊後、大命降下を受けて内閣総理大臣に就任する予定になったが、政局の運営に見通しが立たないと判断すると、総理大臣就任を辞退して桂太郎に地位を譲った（なお、第2次伊藤内閣の内相時代に伊藤が交通事故に遭ったために1ヶ月余り首相臨時代理を務めている）。