Drafted by the New York Yankees in the second round of the 1974 Major League Baseball draft, Bergman was a batting champion and league MVP in each of his first two minor league seasons, first with the New York–Penn League in 1974 and then with the Eastern League in 1975. He played in only 12 games with the Yankees between 1975 and 1977, before being traded to the Houston Astros in December 1977. In four years with the Astros from 1978–1981, Bergman was a part-time player who never had more than 186 at bats or one home run in a season.