In 2009, Whitman's wife of 67 years, Alma Geraldine "Jerry" Crist Whitman, born in Kansas, died at the age of 84. She was a songwriter and embroiderer, daughter of church minister, A.D. Crist. Together they had a daughter, Sharron Beagle (b. 1942), and a son, Byron Keith Whitman (b. 1957). They also had two grandchildren and two great-grandchildren. Byron is a performer and music producer, has released a number of recordings, and toured and recorded with his father on numerous occasions.