During the war Aubrey started to have health problems. He had a bad fall on Christmas Day 1940. Towards the end of the war he started to get blurred vision, his speech became slurred and he had strange feelings in his fingers and toes. He retired in 1945 after the first rehearsal for the Prom season. He still performed occasionally, usually together with his son Dennis. He died on 20 September 1955, three days after a bad fall which gave him a heart attack.