Schuerholz was born in , the son of John Schuerholz Sr., who played in the Philadelphia Athletics minor league system from 1937 to 1940. He is a graduate of the Baltimore City College High School, Towson University and Loyola University. While at Towson, Schuerholz applied for officer candidate school and was rejected, as he was partially deaf. Before his career in baseball, Schuerholz was a teacher at North Point Junior High in Baltimore. Upon leaving his teaching job, he was drafted by the United States Army to serve in the Vietnam War. After entering Major League Baseball with the Baltimore Orioles, Schuerholz joined the United States Army Reserve.