During the Edo period, as tozama daimyo (daimyo who had not been vassals of Ieyasu in 1600) it was transferred several times, as follows: the Awaji Sumoto Domain, holding 30,000 koku -> the Iyo Ozu Domain holding 53,000 koku -> the Shinano Iida Domain, holding 5,5000 koku -> Harima Tatsuno Domain, holding 5,3000 koku (later 51,089 koku by division of succession).
江戸時代には外様大名として淡路洲本藩3万石→伊予国大洲藩5万3000石→信濃飯田藩5万5000石→播磨国龍野藩5万3000石（後に分知で5万1089石）と移封された。