Although the family, which once escaped to Ezogashima Islands, recovered the territory by the arbitration of the Muromachi bakufu, it had to withdraw again to Ezo, and afterwards, the family went down to Dewa Province and finally occupied the west half of the farthest northern area of Dewa Province to the southern area of Ezo, with its headquarters in Hiyama Country (Dewa Province) (present Noshiro City, Akita Prefecture).
いったん蝦夷島に逃れた後室町幕府の調停で復帰したものの再度蝦夷地に撤退、更に出羽に移り檜山郡(出羽国)（現秋田県能代市）を中心に出羽最北部西半から蝦夷地南部を領した。