Gardner went to school at Eagle House and Wellington College and studied at Exeter College, Oxford. He enjoyed jazz, but concentrated on classical music and soon started to compose. An "Intermezzo for Organ" was published by Oxford University Press in 1936. In the same year he got an organ scholarship to Exeter College, Oxford. He got to know the composer Arthur Benjamin and dedicated his "Rhapsody for Oboe and String Quartet" (1935) to him. At Oxford he became friendly with Theodor Adorno with whom he played piano duets.