The Marquess married secondly Maria Elizabeth Clarke, widow of Charles John Clarke, second daughter of Hon Charles Tollemache, of Harrington, Northants, (by his second wife, Gertrude Florinda Gardiner, daughter of Gen William Gardiner), 3rd son of John Manners MP of Hanby Hall, Lincs. Gertrude Florinda Gardiner was also a granddaughter of Louisa Tollemache, 7th Countess of Dysart). They were married on 20 August 1833 at Ham House, Petersham, Surrey. They had one son, Lord Charles William (1834 –1897), a soldier and courtier. She died at Petersham on 7 May 1893, aged eighty-three. On his death in January 1856 at Tottenham Park, his titles passed to his eldest son, George. He was buried at Great Bedwyn church yard. His will was proven in July 1856.