Hollings was married to Martha Patricia Salley until they divorced. Then he was married to Rita Liddy from 1971 until her death in 2012. He had four children with Patricia. He lived in Charleston, South Carolina.