Orr was born in Ann Arbor, Michigan. He was raised in Evansville, Indiana. He studied at Yale University and at Harvard University. Orr was married to Josie Wallace from 1946 until they divorce in 2000. In 2001, he married Mary Davis. Their marriage lasted until his death in 2004.