His grandfather Yoshiyuki SHOZAEMON served Mitsukuni MITO as a dozoban (storehouse keeper), but he was involved in a theft case while at work and got fired, and went back to Takanashi County, Kai Province.
祖父庄左衛門義之は水戸光圀に土蔵番として仕えたが、勤務中に盗難事件に巻き込まれ職を免ぜられてしまい、甲斐国高梨郡に移った。