The family started when his ancestor, whose mother's side of the family was a kugyo (court noble) of the Reizei family, took the name on the mother's side and called himself the Reizei family.
母方を公卿の冷泉家とする祖先が、母姓を冒して冷泉姓を称したことにはじまる。