However, soon he fell out with the Triumvirate and was confined by them for a time, and on April 5, 1567, he escaped from the Triumvirate, allied himself with Hisahide and fought against the Triumvirate in Nara, where they had Todai-ji Temple burnt down in the battle on November 20, 10 (warrior clashes at the Great Buddha Hall of Todai-ji Temple).
しかし、やがて三人衆と不仲になって一時拘禁されたが、永禄10年（1567年）2月16日、三人衆のもとから逃れて久秀と手を結び、奈良で三人衆と戦い、10月10日の戦いで東大寺を焼いてしまった（東大寺大仏殿の戦い）。