Kefauver was born in Madisonville, Tennessee, to Robert Cooke Kefauver and Phredonia Bradford (née Estes). Robert Kefauver was a hardware manager. Estes attended the University of Tennessee from 1922 to 1924, receiving a Bachelor of Arts degree and being initiated into the Kappa Sigma Fraternity. After a year of teaching mathematics and coaching football at a Hot Springs, Arkansas, high school, he attended Yale Law School, where he received a LL.B. "cum laude" in 1927. The next twelve years Kefauver practiced law in Chattanooga, first with the firm of Cooke, Swaney & Cooke, as a partner in Sizer, Chambliss & Kefauver, and later in the firm of Duggan, McDonald, & Kefauver. In 1935 he married Nancy Pigott, who is from Glasgow, Scotland, United Kingdom, whom he had met during her visit to relatives in Chattanooga, Tennessee. They had four children, one of them was adopted. Mrs. Kefauver died in 1967.