In December 1962, Charles married Verity Joan Aldridge in Johannesburg, South Africa. They met three years previously at the Commonwealth Golf tournament when it was held in South Africa. The golfers Denis Hutchinson and Bobby Verwey served as groomsmen. They had two children, Beverly and David. David is a golf director in the United States. He is a successful golf course designer having had major input into the Formosa Country Club east of Auckland, Millbrook at the resort town of Queenstown, and The Dunes course at Matarangi on the Coromandel Peninsula. He was also consultant to the Clearwater course, near Christchurch, designed by golf architect John Darby.