Perry started out as a singer in Special Services after college at Penn, working on Armed Forces Radio during the Korean War. After the service, he worked for a short time at General Electric and then replaced Eddie Fisher as the staff vocalist at Grossingers in the Catskill Mountains and later did comedy working with Sid Caesar as his straight man for several years (which included a four-year stint with Caesar in Las Vegas). Perry appeared in three syndicated specials with Caesar titled "As Caesar Sees It" which aired during the 1962–63 season. These were under his birth name of Jim Dooley. Due to a name conflict with AFTRA, he adopted his mother's maiden name of Perry shortly after beginning his television work.