Roberts was born on September 13, 1941 in Miami, Florida to Henry Melvin Roberts, an appliance repairman, and Edna Wilcher Roberts, a homemaker. His younger sister Cheryl was born in 1947. During World War II, while his father was in the Army, Roberts and his mother lived on the Wilcher family farm in Wheeler County, Georgia. After the war, the family returned to Miami, but Roberts would spend his summers with his grandparents in rural Georgia. Roberts' father had an appliance repair business in Miami.