He had no child before he married Tokiwa gozen (the mother of Zenjo ANO, Gien and MINAMOTO no Yoshitsune) who had been a concubine of MINAMOTO no Yoshitomo as a second wife, ordered by TAIRA no Kiyomori.
長く子がなかったが、源義朝の側室であった常盤御前（阿野全成・義円・源義経の母）を平清盛から下げ渡され、後添えとする。