Edelman was born in Brooklyn, New York. Before becoming an actor, Edelman studied to become a veterinarian at Cornell University, but left during his first year. After serving in the United States Army as an announcer for Armed Forces Radio, he enrolled in Brooklyn College as a theater student, but eventually dropped out. He later worked as a hotel manager and as a taxicab driver. One of his fares was director Mike Nichols, who in 1963 cast Edelman in his breakthrough Broadway role, as the bewildered telephone repairman in Neil Simon's "Barefoot in the Park". Edelman reprised his role in the 1967 film version (starring Robert Redford and Jane Fonda).