After the Western Army lost the battle and the territory of the Uesugi Clan was reduced to 300,000 koku and relocated to the Domain of Yonezawa, Toshimasu served the Domain of Yonezawa and retired to Domori near Yonezawa (currently known as Keijisimizu in Domori, Bansei-Town, Yonezawa-City).
西軍敗退により上杉氏が30万石に減封され米沢藩に移されると、これに従って米沢藩に仕え、米沢近郊の堂森（現、米沢市万世町堂森で慶次清水と呼ばれる）に隠棲した。