Born and raised in Hammersmith, Frankel lived in London for many years, most notably at 17 Soho Square between 1953 and 1957, where he was the host of a circle of artists including the poet Cecil Day Lewis, film director Anthony Asquith, and the writer Leonard Woolf. In 1958 he re-located to Locarno in Switzerland. He married three times: first in 1932 to Joyce Stanmore Rayner (divorced 1944), then to Phyllis Anna Leat (1944 until her death in 1967), and finally to Xenia Hamilton-Kennaway in 1972, not long before his death. There were two sons and one daughter by the first marriage. Frankel died in London on 12 February 1973 while working on the three-act opera "Marching Song" and a ninth symphony, which had been commissioned by the BBC. When he died, "Marching Song" had been completed in short score; it was orchestrated by Buxton Orr, a composer who had studied with Frankel and whose advocacy has been at least partly responsible for the revival of interest in his works.