After that, ownership was trandferred to his daughter, FUJIWARA no Kiyoko (Kokamonin), and then to his heir, Motozane KONOE, but in December, 1166, when Imperial Prince Norihito, the crown prince (and later Emperor Takakura), lived there, Higashi Sanjo-dono Residence caught fire and was destroyed, and had been in ruins thereafter without being reconstructed.
その後、忠通の娘・藤原聖子（皇嘉門院）、続いて嫡男の近衛基実に譲られたが、仁安元年（1166年）12月に当時皇太子憲仁親王（後の高倉天皇）の御所であった東三条殿は火災で焼失、以後は再建されずに荒廃した。