After 1956, Baker's success was limited. His most notable victory was a sound unanimous decision defeat of solid Canadian George Chuvalo, the future Canadian heavyweight champion, by unanimous decision on September 9 1957. Chuvalo was also a multiple time championship contender in the 1960s and 1970s.