However, subsequently, he became chased by Yoritomo who had re-established his influence, and then, after the Battle of Fujigawa, he was captured and sent to his adopted son-in-law, Yoshizumi MIURA.
しかしやがて勢力を盛り返した頼朝によって逆に追われる身となり、富士川の戦いの後捕らえられ、娘婿の三浦義澄に預けられる。