Lee was born in Wheeling, West Virginia. In 1927, he moved with his parents to Cleveland, Ohio. He married Sylvia Olden Lee in 1944. The couple later divorced. He later married his second wife. He lives in Sweden. He turned 100 in 2016.