His father Yukitsuna was the heir of the Tada-Genji (Minamoto clan), but he was impoverished after his family's hereditary estate, Tada no sho, was confiscated during the political purge carried out by MINAMOTO no Yoritomo.
多田源氏の惣領であった父行綱は、源頼朝の粛清により累代の所領多田荘を奪われ没落。