While in college, Carroll had received training through the Air Force Reserve Officer Training Corps. By graduation, he had risen to the rank of Commandant of Cadets, the highest rank of any student at the university. After graduation, he enlisted in the Air Force and was stationed at Carswell Air Force Base in Fort Worth, Texas. For three years, he served as an Air Force attorney, then returned to Paducah and joined the law firm of Reed, Scent, Reed, and Walton. He was active in civic affairs, including membership in the Jaycees and serving as charter president of the Paducah Optimists Club in 1962. He was a frequent lay speaker in the Cumberland Presbyterian Church, and from 1966 to 1967, served as moderator for the Kentucky Synod.