Though recognized as Ujiharu's son, he was also an illegitimate child, and as such, when Ujiharu concluded his alliance with Ujiyasu HOJO, Tomoharu was sent to the Hojo clan as hostage, eventually becoming a vassal of the Hojo clan; Tomoharu also served as the replacement for Ujinao HOJO.
氏治の子であるが、庶子であったため、氏治が北条氏康と同盟を結んだとき、その人質として後北条氏に赴いて、そのまま家臣となり、北条氏直の代も仕えた。