Despite his promotion, according to a letter sent to Alice BACON by his sister Sutematsu YAMAKAWA and in retrospect by Shibagoro, the persons formerly concerned with Aizu Domain always lodged at this former chief retainer's place, and also some of his relatives pestered him for allowance like a kind of blackmailer, so he was always struggling with financial difficulty until his later years.
このように栄進した浩ではあったが、妹の山川捨松がアリス・ベーコンに送った手紙や柴五郎の回顧などによると、元家老であった浩の元には常に元会津藩関係者が寄宿しており、また、出世した浩に対して一種のたかりのように仕送りをせがむ親戚もいたようで、晩年まで生活は非常に苦しかったという。