From 1934 to 1937, Gorey attended public schools in the Chicago suburb of Wilmette, Illinois, where his classmates included Charlton Heston and Warren MacKenzie. Some of his earliest preserved work appears in the Stolp School yearbook for 1937. After that, he attended the Francis W. Parker School in Chicago. He spent 1944 to 1946 in the Army at Dugway Proving Ground in Utah. He then attended Harvard University, beginning in 1946 and graduating in the class of 1950; he studied French and roomed with poet Frank O'Hara.