In contrast to the Taira clan, who had been given high-ranking posts by the Retired Emperor Shirakawa and Emperor Toba, Tameyoshi was given the cold shoulder; having had his request to serve as Governor of Mutsu Province, a post served by his father, MINAMOTO no Yoriyoshi, and his grandfather, MINAMOTO no Yoriie, turned down, he had to serve as Saemon no jo for more than thirty years.
平氏が白河院、鳥羽天皇に重用されるのに対して、為義は源頼義、義家と父祖代々任じられた陸奥国守任官の要望も却下され、30余年もの間、左衛門尉のまま冷遇された。