In 1884, Toshikazu was ennobled as a marquess in the new peerage in honour of his father's achievements. He married Shigeno Naoko (1875–1918), but had no children and relinquished the title in 1928 in favour of his younger brother Toshitake (1865–1943). A graduate of Yale and Heidelberg universities, Toshitake successively served as governor of Tottori (1900), Ōita (1901–1905), Saitama (1905–1907) and Osaka (1912–1917) prefectures. He married Kondō Sakae (1879–1956) and had three children, Toshiaki (1900–1995), Toshimasa (1902–1945) and Michitada (1907–????). Toshiaki became a prominent professor of Japanese history and succeeded as the 3rd Marquess in 1943, holding the title until the peerage was abolished in 1947. He subsequently became the librarian of the National Diet Library from 1951 to 1953 and then taught as a lecturer and professor of history at Nagoya University (1953–1959) and at Rikkyū University (1959–1965). He was awarded the Asahi Prize in 1993, two years before his death. He married Yoneda Yaeko (1910–????) and had two children, Yasushi (b. 1934) and Shigeko (b. 1936). Yasushi married Matsudaira Naoko (b. 1940) and had a daughter, Akiko (b. 1965).