Lees father was a painter in Taiwan. His mother was an elementary school teacher, and his elder brother, Yuan-Chuan Lee, has been a professor in at Johns Hopkins University for 40 years, awarded the honor Special Chair Lectureship in Academia Sinica in Taiwan. Besides, his younger brother, Yuan-Pern Lee, also awarded this honor. Lees sister, Chi-Mei Lee has served as a professor in National Chung Hsing University.
In 2003, he was one of 22 Nobel Laureates who signed the Humanist Manifesto.