Haller was born in Lockport, Illinois and attended the University of Illinois, where he played as a quarterback for the Illinois Fighting Illini football team. During his time at the university, Haller was also a member of Theta Chi fraternity. Haller was signed by the San Francisco Giants, as an amateur free agent, in 1958. After playing in the minor leagues for three seasons, he made his major league debut with the Giants on April 11, 1961, at the age of 24.