Hoff was born in Bronx, New York. While he was still at high school, Milt Gross, a popular 1930s cartoonist, told him at an assembly, "Kid, someday you'll be a great cartoonist!" At 16, he enrolled at the National Academy of Design in New York City. At 18, he sold his first cartoon to "The New Yorker", and eventually sold a total of 571 of them to the publication from 1931 to 1975. Hoff became known for his cartoons in "The New Yorker" depicting tenements and lower-middle class life in the city.