Huo was born in Xiaonanhe Village in Jinghai County, Tianjin, as the fourth of Huo Endi's ten children. The family's main source of income was from agriculture, but Huo Endi also made a living by escorting merchant caravans to Manchuria and back. Although he was from a family of traditional wushu practitioners, Huo was born weak and susceptible to illness. He had asthma and, at an early age, he contracted jaundice, which would recur periodically for the rest of his life. It is theorised that he may have had a mild form of congenital jaundice known as Gilbert's syndrome. Due to his frail frame, his father discouraged him from learning wushu.