He had several children including a biological son, Harusue SAIONJI who was Konoefu (Government Official at the Headquarters of the Inner Palace Guards) and an adopted child, Kinzumi SAIONJI who was a biological son of Imperial Prince Arisugawanomiya Tsunahito.
実子に近衛府西園寺治季、養子縁組に近衛府西園寺公潔（有栖川宮韶仁親王の王子）がいる。