Takanobu was raised by FUJIWARA no Toshinari who was the husband from a remarriage by Takanobu's mother (FUJIWARA no Teika, a tanka poet, was a half younger brother of Takanobu with the same mother).
母の再婚相手である藤原俊成に育てられる（歌人・藤原定家は異父弟にあたる）。