When his foster mother of the Katsura family died in 1841, he returned to the Wada family where he was born, and grew up with his biological parents and elder sister.
翌年、桂家の養母も亡くなったため、生家の和田家に戻って、実父、実母、次姉と共に育つ。