Lee was born in the rural farming community of Sanshi Village, Taihoku Prefecture, Japanese Taiwan. He is of Yongding, Tingzhou Hakka descent. As a child, he often dreamed of traveling abroad, and became an avid stamp collector. Growing up under Japanese colonial rule, he developed a strong interest for Japan. His father was a middle-level Japanese police aide, and his elder brother, Lee Teng-chin (李登欽) joined the colony's police academy and soon volunteered for the Imperial Japanese Navy and died in Manila. Lee—one of only four Taiwanese students in his class—graduated with honors and was given a scholarship to Japan's Kyoto Imperial University.