Gregory was born on October 12, 1932 in St. Louis, Missouri. He was raised in Chicago, Illinois. Gregory studied at Sumner High School and at Southern Illinois University Carbondale. He had been married to Lillian Gregory since 1959. They had 10 children.