His son Victor, the 2nd Marquess of Linlithgow became the longest-serving Viceroy of India 1936-43, a job he had always wanted. His grandson Lord Glendevon married the daughter of the English novelist W. Somerset Maugham.