His father was Gon Dainagon (provisional chief councilor of state) Akimichi KAJUJI; his mother was a daughter of Tsunemichi INABA who was Hida no kami (the governor of Hida Province) and the lord of the Usuki Domain.
権大納言・勧修寺顕道の子、母は飛騨守・稲葉恒通（臼杵藩主）の娘。