He was from the lineage that began from the Prince MINAMOTO no Yoshiari of Emperor Montoku, and his father Suenori was a northern bushi (samurai warrior) that served Emperor Toba.
文徳天皇の皇子源能有に始まる家柄で、父の季範は鳥羽天皇に仕える北面武士だった。