He had KOSE no Iratsume and ISHIKAWA no Uchimyobu as his wives, and OTOMO no Tabito, OTOMO no Tanushi, OTOMO no Sukunamaro, OTOMO no Inakimi and OTOMO no Sakanoue no Iratsume as his children.
妻には巨勢郎女、石川内命婦、子には大伴旅人、大伴田主、大伴宿奈麻呂、大伴稲公、大伴坂上郎女がいる。