Torn was born on February 6, 1931 in Temple, Texas. He studied at Texas A&M University. Torn was married to Ann Wedgeworth from 1955 until they divorced in 1961. Then he was married to Geraldine Page from 1963 until her death in 1987. He married Amy Wright in 1989. He had six children.