Sherman was born in New York City to Russian Jewish immigrants, Rosa (Dancis) and Al Sherman. In 1953 Sherman was drafted into United States Army ans assigned to the Army Band and glee club. He was musical conductor for both groups from 1953 until his honorable discharge in 1955. He studied at Bard College.