Munetoshi's first son Yoshikatsu YAGYU had served Hisahide and in the battle against Junkei TSUTSUI, he had been shot and badly injured; the second son Kyusai YAGYU and the third son Tokusai YAGYU had become priests; and the fourth son Muneaki YAGYU had served Kazuuji NAKAMURA; so when Ieyasu requested Munetoshi to become an instructor, only Munenori happened to be available for serving the Tokugawa clan.
徳川氏に仕えることとなったのは、長男の柳生厳勝は久秀配下として筒井順慶と戦ったとき、鉄砲により戦傷を負い、次男の柳生久斎と三男の柳生徳斎は僧侶となり、四男の柳生宗章は中村一氏に仕官していたからである。