Lee Wai Tong was born in Tai Hang, Hong Kong as the third child to a construction firm owner before he moved back to his parents home of Wuhua County in Meizhou, Guangdong, China at the age of four. His father is born from Hong Kong while mother is Chen Qiongsheng, Guangdong Xiangshan (now Zhongshan City). It was there that he was informally taught how to play football until he moved back to Hong Kong and was formally trained at Queen's College, Hong Kong. He left school early initially to help his fathers construction company before he joined South China AA as a youth player in 1922. By 1923 he would graduate to the senior team where he quickly showed himself as being a highly talented striker, which soon saw him called up to the Chinese national football team to represent them for the 1923 Far Eastern Championship Games held in Osaka, Japan. Within the tournament China won the championship and Lee was considered the young stand out star of the tournament.