Yorinaga married the eldest daughter of Saneyoshi TOKUDAIJI, Koshi (the aunt of Masaruko), and lived in Oimikado Takakura tei with Tokudaiji family members; he adopted the daughter of his brother in law, Kinyoshi when the daughter was young.
頼長は徳大寺実能の長女・幸子（多子の伯母）と結婚して、徳大寺家の人々と大炊御門高倉邸に住んでいたことから、義弟・公能の娘を幼い頃から養女としていた。