His legitimate wife was a daughter in the Kutsuki clan and gave birth to Yoshinari SUGIYAMA and Kamon ISHIDA, and his second wife was a daughter in the Tsuge clan and gave birth to Kahei Nariyasu SUGIYAMA.
妻は朽木氏の娘、子に杉山吉成、石田掃部、後妻は柘植氏の娘、子に杉山嘉兵衛成保。