During the 1950s Baker played in several big bands, including Lionel Hampton's orchestra. After moving to California in 1956, he played with the West Coast jazz orchestras of Stan Kenton and Maynard Ferguson before returning to Indianapolis to lead his jazz band for two years. He performed in clubs across the United States, including the Five Spot Café in New York City with George Russell in the late 1950s. In 1960 he toured Europe as a member of Quincy Jones's band. He also performed in Canada, Japan, Australia, and New Zealand during his more than sixty-year career.