When in August, 1180, MINAMOTO no Yoritomo raised an army, Shigenari took the side of the Heike (Taira family) and fought against Yoritomo; in October of the same year, he came under Yoritomo's control and became Yoritomo's gokenin along with the Chichibu family such as Shigetada HATAKEYAMA at a place called Nagai no Watashi along the Sumida-gawa River.
治承4年（1180年）8月の源頼朝挙兵では平家方として頼朝と敵対したが、同年10月、隅田川の長井の渡しにおいて、畠山重忠ら秩父一族と共に頼朝に帰伏し御家人となる。