His father came from the Rinno family, the chief retainer of the Obama Domain in Wakasa Province, and it is said that his father changed his name to 'Koishi' while leading an itinerant life.
父は若狭国小浜藩家老林野氏の出自で、流浪中に小石姓に改めたという。