Callicott was born in Memphis, Tennessee on May 9, 1941, to distinguished regional artist and art instructor Burton H. Callicott (1907–2003), of the Memphis Academy of Arts (now Memphis College of Art). In 1959, Callicott graduated from Memphis's then racially segregated Messick High School and attended Southwestern at Memphis (now Rhodes College), earning a B. A. in philosophy with Honors in 1963. He received a Woodrow Wilson Fellowship for graduate study at Syracuse University, completing his M. A. in philosophy (1966) and his doctorate in the same field (1972) after earning a Woodrow Wilson Dissertation Fellowship. His dissertation, titled "Plato’s Aesthetics: An Introduction to the Theory of Forms", drew from the concentration of his undergraduate and graduate work: ancient Greek philosophy.