Although he was expected to restore order in the capital that had been devastated by the long years of famine and Heike clan's violence he came to be on bad terms with Monk-Emperor Go-Shirakawa because of his failure to restore order, worsening food situation caused by the vast army staying in the capital and his intervention in the succession of the Imperial Throne.
長年の飢饉と平家の狼藉によって荒廃した都の治安回復を期待されたが、治安回復の失敗と大軍が都に居座ったことによる食糧事情の悪化、皇位継承への介入などにより後白河法皇と不和となる。