According to "Hankanpu" (Genealogy of the Protectors of the Shogunate) compiled by Hakuseki ARAI and "Kansei Choshu Shokafu" (A record of family trees of samurai warriors of the Tokugawa Shogunate) which was published in later years, the original family name of the Yagyu clan was Sugawara, and it is said that they were descendants of SUGAWARA no Michizane.
新井白石が作成した系譜の『藩翰譜』（または後世の『寛政重修諸家譜』）によると、柳生氏の姓は菅原氏とされ、菅原道真が祖先とも言われている。