When Zimmer was 12 years old, his mother married Howard Rubin, a Korean War veteran with three children of his own. The newly combined family moved to Glen Ridge, New Jersey, and Rubin worked at the post office there. Zimmer attended Glen Ridge High School, where he was selected as the class speaker for his graduation ceremony. His mother, suffering from lymphoma, required paramedics to take her from Columbia Presbyterian Hospital to the school auditorium on a stretcher to hear the address. She died several days later.