Baron Moriteru NAGAYAMA, who served as a Member of the House of Peers, Provisional Governor of Chikuma Prefecture, and Governor of Niigata Prefecture, was his elder brother.
貴族院議員・筑摩県権県令・新潟県令等を務めた永山盛輝男爵は兄。