As a child of Empire, from his mid-teens onwards, Butler was expected to look after his younger siblings, arranging for them to stay with relatives during school holidays and sending them Christmas presents that he pretended had been sent by their parents. His sister was the writer Iris Mary Butler (born 1905), who became Iris Portal upon her marriage, and her elder daughter is Jane Williams, Baroness Williams of Elvel, the mother of Justin Welby, the current Archbishop of Canterbury. Butler's younger brother Jock, a Home Office civil servant and Pilot Officer, was killed in a plane crash at the end of 1942.