In 1980, Olsen moved to 16-time Belgian champions R.S.C. Anderlecht, to play alongside Danish internationals Benny Nielsen and Kenneth Brylle. In his first year at Anderlecht, the club won the 1980–81 Belgian First Division, conceding only 24 goals in 34 games. It was for Olsen the first trophy of his senior career. He played six years at Anderlecht, winning three Belgian championships with the club, and he was eventually named Anderlecht team captain. During most of 1982, Olsen suffered from a severe shin injury, which prompted Anderlecht manager Tomislav Ivic to move him back from the midfield into the libero position. As the libero at Anderlecht, Olsen controlled an aggressive form of off-side trap, which had 3–4 players converge towards the ball-possessing player as the off-side trap was sprung, in order to prevent the ball-possessing player from countering the off-side by dribbling on his own.