Griffith was born on June 1, 1926 in Mount Airy, North Carolina, the only child of Carl Lee Griffith and his wife, Geneva (née Nunn). He studied at the University of North Carolina at Chapel Hill. He was the father of two children. He was married three times.