Emperor Shirakawa started to use Kakuen's once-used living quarters as his Gosho (Imperial Palace) (Shirakawa Izumi-dono) around 1090 after his retirement, then in 1115, erected the Shirakawa Minami-dono (South Imperial Palace) rebuilding the Shirakawa Izumi-dono, further in 1118, at the adjoining site, erected the Shirakawa Kita-dono (North Imperial Palace), and conducted the Insei (rule by the Retired Emperor) shuttling between the two Palaces.
また、白河天皇は退位後の寛治4年（1090年）頃に旧の覚円の僧房を御所（白河泉殿）とし、続いて永久(日本)3年（1115年）に白河泉殿を改築して白河南殿を造営、更に元永元年（1118年）にはその隣接地に白河北殿を新造して2つの御所を行き来しながら院政を行った。