He was the second child of Kiyofusa KAJUJI (a descendant of FUJIWARA no Takafuji of the Northern House of the Fujiwara clan), a mid-ranking court noble from Kyoto who held the position of Saemon no jo (Lieutenant of the Left Division of the Outer Palace guards).
京都の中級公家であった、左衛門尉勧修寺清房（藤原北家藤原高藤流）の二子として生まれる。