Yang was born in Hefei, Anhui, China; his father, Ko Chuan Yang (; 1896–1973), was a mathematician, and his mother, Meng Hwa Loh Yang (), was a housewife. Yang attended elementary school and high school in Beijing, and in the autumn of 1937 his family moved to Hefei after the Japanese invaded China. In 1938 they moved to Kunming, Yunnan, where National Southwestern Associated University (Lianda), was located. In the same year, as a second year student, Yang passed the entrance examination and studied at Lianda. He received his bachelor's degree in 1942, with his thesis on the application of group theory to molecular spectra, under the supervision of Ta-You Wu. He continued to study graduate courses there for two years under the supervision of Wang Zhuxi, working on statistical mechanics. In 1944 he received his master's degree from Tsinghua University, which had moved to Kunming during the Sino-Japanese War (1937–1945). Yang was then awarded a scholarship from the Boxer Indemnity Scholarship Program, set up by the United States government using part of the money China had been forced to pay following the Boxer Rebellion. His departure for the United States was delayed for one year, during which time he taught in a middle school as a teacher and studied field theory.