Despite a low-key club career, Madsen's international performances for the Denmark national football team got the attention of European professional clubs. As an amateur player, Madsen was a free agent able to sign with any professional club without the need of a transfer fee. In 1963, a contract with Italian club Atalanta Bergamo was hindered by a broken nose and subsequent hospitalization. Madsen was offered a contract by Spanish club FC Barcelona in 1964, but he turned it down in order to stay an amateur with the Danish national team and HIK in the Danish 3rd Division. In 1965, Madsen signed a professional contract with Dutch team Sparta Rotterdam, where he enjoyed success. Madsen stayed three seasons with Sparta, with whom he won the 1966 KNVB Cup. Madsen scored the only goal, as Sparta beat ADO Den Haag 1–0 in the KNVB Cup final. In the Dutch Eredivisie championship, Ole Madsen scored a total 22 goals in 71 games for Sparta.