On the breaking of the Onin War, he was forced to retire on orders of the Kamakura bakufu (Japanese feudal government headed by a shogun) because he was the father-in-law of the Imperial Prince Masanari, the Prince of Emperor Gotoba.
承久の乱の折に後鳥羽天皇の皇子・雅成親王の義父だった事から、鎌倉幕府から隠居を命じられる。