There was a theory that Kagemori had been an illegitimate child of MINAMOTO no Yoritomo, and thereafter, this theory indirectly caused the killing (of someone who is guilty) of the family in the Shimotsuki sodo (the political change of Kamakura Bakufu, the shogunate government) in the period of Kagemori's grandson, Yasumori ADACHI.
また源頼朝御落胤説があり、これが後に孫の安達泰盛の代になり、霜月騒動で一族誅伐に至る遠因となる。