He was born as the third son of Sadamoto, then the head of the SERIZAWA family; this family originated from a powerful family named SERIZAWA that arose during the middle ages in Serizawa Village, Hitachi Province, became a vassal of the Tokugawa Shogunate through distinguished services in the Battle of Sekigahara and subsequently became a Senior Goshi samurai of Mito Clan.
中世に興起した常陸国芹沢村（現茨城県行方市玉造町芹沢）の豪族、芹沢氏から発祥し、関ヶ原の戦いの戦功により幕臣となり、のちに水戸藩上席郷士（士分）となった芹沢家の当主貞幹の三男として生まれた。