Fisher was born on August 10, 1928 in Philadelphia, Pennsylvania to a Russian-Jewish family. He studied at Thomas Junior High School, South Philadelphia High School, and at Simon Gratz High School. It was during high school many people discovered that Fisher had a talent for singing. 