The next year, his father joined the Shogitai (group of former Tokugawa retainers who opposed to the Meiji government and fought in the Battle of Ueno) to take the side of the Shogunate, and fight against Meiji government troops first in Ueno and later in Hakodate, while Aizan moved to Shizuoka with his grandparents.
翌年、父は幕府方として彰義隊に加わり、上野に籠り、のち函館に移って政府軍と戦い、かれは祖父母とともに静岡に移った。