At the time of the Hogen Rebellion, Tameyoshi transferred responsibility for the family to Yoshitomo (although another theory states that Tameyoshi renounced Yoshitomo and transferred the position of head of the family to the fourth son MINAMOTO no Yorikata.)
保元の乱当時は、義朝に家督を譲っていた（一説に義朝を勘当し、四男の源頼賢に家督を譲っていたともいう）。