However, the political structure and financial/military support at that time were insufficient to maintain such a situation, and because many Emperors were young and short-lived in the middle of the Heian period, there was no Joko with the energy and health needed to exercise sufficient authority; consequently, this structure by which authority was exercised by the Emperor's father gradually declined and was eventually replaced by a system called Sekkan seiji (摂関政治), in which a member of the Northern House of the Fujiwara clan exercised rights and performed duties in place of the Emperor, became widespread.
だが、当時はまだこの状態を常に維持するための政治的組織や財政的・軍事的裏付けが不十分であり、平安時代中期には幼く短命な天皇が多く十分な指導力を発揮するための若さと健康を保持した上皇が絶えて久しかったために、父系によるこの仕組みは衰退し、代わりに母系にあたる天皇の外祖父の地位を占めた藤原北家が天皇の職務・権利を代理・代行する摂関政治が隆盛していくことになる。