The social standing of Yasuba family was Karo (chief retainer) of Hosokawa Domain, and Yasuba family had so enormous influence in the domain that one of the ancestors Ippei YASUBA was entrusted with carrying out Kaishaku (assistance in someone's committing hara-kiri by beheading him) of Kuranosuke OISHI, Hitto karo (head of chief retainers) of Asano family of Ako Domain who had been left to Hosokawa Domain, in the hantei (residence maintained by a daimyo in Edo) of the Hosokawa Domain in Takanawa, Tokyo (because of this relationship, Yasuba family has kept the deep knowledge about Genroku Ako Incident, and the present head of Yasuba family Mr. Yasumasa YASUBA is acting as Kaicho (Chairman) of Zenkoku Gishikai Rengokai (National combination of loyal retainers societies).
細川藩家老の家柄で、先祖である安場一平は、東京高輪の細川藩邸で、細川藩預かりとなっていた赤穂藩浅野家筆頭家老大石内蔵助の介錯を行うなど（この縁で安場家は元禄赤穂事件に造詣が深く、現在の安場家当主安場保雅氏は全国義士会連合会の会長をつとめている）、藩内でも影響力のある家柄であった。