After that, its ownership was transferred for a while to FUJIWARA no Kanshi (the Empress of the Emperor Goreizei), his daughter, but around 1060, it became customary that important events of imperial families and sekkan-ke (the top family in the govermental hierarchy), such as big banquets by sekkan-ke, the ceremony of assuming the position of Toshi Choja (the chief of the Fujiwara clan), the ceremonies of instituting a crown prince and an empress and coming-of-age ceremonies, were held here, and therefore, it was decided after the era of FUJIWARA no Moromichi that the place would be owned by Toshi Choja (the chief of the sekkan-ke) in each generation.
その後、一時期頼通の娘・藤原寛子（後冷泉天皇皇后）に譲られたものの、康平3年（1060年）頃より以後の摂関家の大饗や藤氏長者就任の儀式、立太子や立后、元服など、摂関家及び皇室の重要行事はここで行われる慣例が形成され、所有者も藤原師通以後は代々藤氏長者（＝摂関家当主）の所有と定められた。