He had elder brothers from a different mother, FUJIWARA no Tsunesada and FUJIWARA no Mitsutada, and an elder sister from the same mother, MINAMOTO no Yoshiko (Emperor Goshirakawa's consort) and a child, FUJIWARA no Yorizane.
異母兄に藤原経定・光忠、同母姉に源懿子(後白河天皇妃)、子に藤原頼実がいる。