An expert marksman with the bow, Yoriyoshi earned high praise for his bravery starting when he was still young, and in 1030 he and his father Yorinobu took up arms to suppress TAIRA no Tadatsune War (also called the Chogen War); thereafter, he took over the headship of the Kawachi-Genji from his father Yorinobu, and began planning to relocate the seat of his power in eastern lands.
弓の達人で若い頃から武勇の誉れ高く、長元3年（1030年）には父の頼信とともに平忠常の乱（長元の乱）を平定し、父頼信の後を継いで河内源氏の東国への進出を図る。