He married Benita Lees (1892–1968), daughter of Sir Harold Pelly, 4th Bt., and the widow of Sir Thomas Lees, 2nd Bt., in 1929. The marriage was very happy for the uxorious Brooke and resulted in one daughter and one son, Kathleen and Victor. During the war the couple lived in Hartley Wintney in Hampshire. After the war, the Brookes' financial situation forced the couple to move into the gardener's cottage of their former home, where they lived for the rest of their lives. Their last years were darkened by the death of their daughter, Kathleen, in a riding accident in 1961.