His father served MINAMOTO no Yoritomo ever since he was exiled in Izu, and he served as a shugo (provincial constable) in four provinces such as Omi Province, and Hirotsuna was appointed to sahyoe no jo (third-ranked officer of Sahyoe-fu, the Left Division of Middle Palace Guards) at an early age as the heir to his father.
父は源頼朝に伊豆の流人の頃から仕え、近江国を始め四ヶ国の守護を務めており、広綱はその嫡男として早くに左兵衛尉に任じられた。