Thornburgh married Ginny Hooton, and they had three sons together (John, David and Peter). Ginny Hooton was killed in an automobile accident in 1960, which left Peter, the youngest of their three sons, with physical and intellectual disability. In 1963 Thornburgh was remarried, to Ginny Judson, with whom he had another son, Bill, in 1966. Ginny (Judson) Thornburgh was a former schoolteacher from New York, who holds degrees from Wheaton College in Norton, Massachusetts, and the Harvard Graduate School of Education. A lifelong advocate of people with disabilities, she served as Director of the Interfaith Initiative of the American Association of People with Disabilities, based in Washington, D.C., and has co-authored and edited "That All May Worship," an award-winning handbook for religious congregations working to include people with all types of disabilities. She received the Hubert H. Humphrey Civil Rights Award from the Leadership Conference on Civil Rights in April 2005.