Perelman was born in Brooklyn, New York, the only son of Joseph and Sophie Perelman, who moved from one failed business to another until they found themselves raising chickens on a farm and running a dry-goods store in Providence, Rhode Island. He attended the Candace Street Grammar School and Classical High School in Providence, Rhode Island. He entered Brown University in 1921, where he became editor of the campus humor magazine "The Brown Jug" in 1924. Perelman dropped out of Brown and moved to Greenwich Village in New York.