On 20 June 1965, Madsen scored an extraordinary goal in an international game against the Sweden national football team. The ball fell to him in an inconvenient angle, with Ole himself facing away from goal, but he hit the ball awkwardly with his heel, and it went past Swedish goalkeeper Arne Arvidsson for a 1–0 lead. Denmark eventually won the game 2–1, the first Danish victory against their neighbors in 14 years. Ole Madsen was then seen as the most popular Danish footballer since Pauli Jørgensen. When Ole Madsen signed a professional contract in 1965, he was automatically banned from the amateur-only national team.