After that, ownership of the castle was transferred from one family to another, from the Nagai family to the Abe family, then the Okudaira family, and to the Aoyama family, but in 1758, Sukemasa MATSUDAIRA succeeded the castle with 770 thousand koku to keep it for seven generations until the Meiji Restoration.
その後は永井家・阿部家・奥平家・青山家と目まぐるしく城主は替わったが、1758年(宝暦8年)に松平資昌が7.7万石で入封し、以後7代続いて明治維新を迎えた。