On 16 December 1920, then-Viscount Stuart married Eleanor May, daughter of Solomon Guggenheim. They had four sons, of whom the two eldest were killed in the Second World War. He committed suicide with a shotgun in his study at Old Lodge, his house and estate at Ashdown Forest near Uckfield on 5 November 1961, aged 72, and was succeeded in his titles by his third son, Arthur Stuart, 8th Earl Castle Stewart.
On 16 December 1920, Viscount Stuart married Eleanor May, daughter of Solomon Guggenheim. The couple had four sons, with the two oldest being casualties in World War II. He met his demise with a shotgun in his study at Old Lodge on November 5, 1961, aged 72, and was succeeded by his third son, Arthur Stuart, who became the 8th Earl Castle Stewart.