Koch was born on December 12, 1924 in The Bronx, New York to Yetta (née Silpe) and Louis Koch. They were a Polish-Jewish family from New Jersey. He was raised in both The Bronx and in Newark, New Jersey. Koch studied at the City College of New York and at New York University School of Law.