There are various theories, but it is believed that he rejected the offer from Ujiyasu HOJO (some say he served the Hojo clan for a period of time) and returned to the service of the Tokugawa clan.
諸説あるが、北条氏康からの誘いを断り（北条氏に一時期仕えたという説もある）、徳川氏に復帰。