Gao Shaoyi was the third son of Emperor Wenxuan, Northern Qi's first emperor. His mother was Emperor Wenxuan's concubine Consort Feng, who carried the title of "Shifu" (世婦), a title for third class imperial consorts. It is not known when he was born. In 559, Emperor Wenxuan created him the Prince of Guangyang. After Emperor Wenxuan's death later that year, Gao Shaoyi's older brother Gao Yin became emperor (as Emperor Fei), and around the new year 560, Emperor Fei changed Gao Shaoyi's title to Prince of Fanyang.