Thielemans started his career as a guitar player. In 1949 he joined a jam session in Paris with Sidney Bechet, Charlie Parker, Miles Davis, Max Roach and others. In 1949 and 1950 he was in European tours with Benny Goodman. He made his first record in Paris with fellow band member, tenor saxophonist Zoot Sims. In 1951 he became a band member of the Bobbejaan Schoepen.