Born in New York City, Drake had his first song published at age 12, in 1931. The son of Jewish immigrants Max Druckman and Pearl Cohen, he attended Townsend Harris High School in the borough of Queens, New York, graduating in 1935, and went on to receive a Bachelor of Arts degree in social science from the City College of New York in 1940. His elder brother, Milton Drake, also became a songwriter, with work including "The Java Jive" and "Nina Never Knew"; and his younger brother Arnold Drake, became a writer for DC Comics, Marvel Comics, and others, as well as an author and playwright.