Polster came through the Austria Wien youth system to make his professional league debut in August 1982, at 18 years of age. He scored his first Bundesliga goal three weeks later and went on to win three league titles and a domestic cup before moving abroad to play a season in Serie A with Torino. He then spent the five following years at Spanish teams Sevilla, Logroñés and Rayo Vallecano, ending up with these teams in mid-table as well except for one year, 1989–90, in which Sevilla FC ended in sixth place and played UEFA Cup the following year. In 1990, he finished runner-up in the Spanish goalscoring chart. In 1993, he moved to Germany to spend five years at Köln, again ending up in mid-table every season except for the last one in which he experienced relegation. That made him join Borussia Mönchengladbach next year but they got also relegated at the end of the season and Polster returned to Austria to play a final season at Austria Salzburg.