As the Southern House of the Fujiwara clan where Tadamori was born already lost their former influence as a result of FUJIWARA no Nakamaro's War and so on while instead the Ceremonial House of the Fujiwara Clan became the mainstream, because the latter were maternal relatives of Emperor Kanmu, Tadamori's grandfather, FUJIWARA no Kosemaro's rank was Sangi (Councilor) and his father, FUJIWARA no Matanari was Goinojo (Fifth Rank, Upper Grade) and Awa no kami (Governor of Awa Province) at best.
彼が生まれた時は南家は既に藤原仲麻呂の乱等でかつての勢いを失い、代わりに藤原式家が桓武天皇の外戚として主流に立っていたことなどから、祖父の藤原巨勢麻呂は参議・父の藤原真作は従五位上阿波守止まりであった。