While growing up Davis was known as Little Pops. His second home was the Phyllis Wheatley settlement house. Its director Gertrude Brown gave the area's children continuing education in a safe place to go after school. Davis must have gotten into trouble at some point as it was a juvenile parole officer who suggested he attend Phyllis Wheatley. There with his friends, Davis learned boxing, etiquette, and Robert's Rules of Order. They met Chick Webb and danced to Duke Ellington and Count Basie who stayed there on tours to the Orpheum Theatre. In 1941, he graduated from North High School where he lettered in and won a city championship in boxing. Davis attended the University of Minnesota and later in life received an honorary doctorate in law from Macalester College.