Although his father, Yoritoki, died in the battle field in 1057, the clan beat the Yoriyoshi army in November through the Kawasaki no saku (stockades of Kawasaki).
天喜5年（1057年）には父の頼時が戦死するものの、11月には河崎柵に拠って頼義勢を撃破。