After 13 straight wins (Baker's dozen), Baker lost to Tommy Hurricane Jackson on February 3 1956, in New York's Madison Square Garden. It was a conytorversial split decision win for Jackson. In May 1956, Baker easily defeated John Holman. Baker also lost another controversial split decision rematch with Jackson on September 26 1956. Both fights should have been draws.Opinions on both Baker-Jackson fights were divided in many publications with most of them giving the fight to Baker. United Press scored the fight 7-5 for Baker. A poll of all 13 ringside writers favored Baker 9-4.