After his first wife died, he married Haruko, a former maiko (apprentice geisha) in Gion, Kyoto whom he had patronised for a long time and had had adopted by Hisamoto HIJIKATA.
死去後、長年囲っていた京都祇園の舞妓出身の春子を土方久元養女として後妻に迎える。