Originally, Shibayama was an another name for the KAJUJI family, and the first family head of the KAJUJI family, Naidaijin (minister of the center) Tsuneaki KAJUJI was also called Naidaijin Shibayama.
芝山とは元々勧修寺家の別称であり、勧修寺家初代、内大臣・勧修寺経顕も芝山内大臣と称した。