The name of the Hildyard family of Flintham was initially Thoroton. Col. Hildyard, father of MP Hildyard, was formerly called Thomas Blackborne Thoroton. The second son of Thomas Hildyard, formerly Thomas Thoroton, took holy orders and became a rector. In 1816 the Rev. Levett Thoroton married in London the daughter of Sir Alexander Cray Grant, 8th Baronet of Dalvey, Elgin, Scotland, and MP. but changed his name to Hildyard in 1815 on marrying a Hildyard heiress, the niece of Sir Robert d'Arcy Hildyard, 4th and last Baronet, who died without issue leaving his estate to his niece. Col. Thoroton Hildyard was descended from Mary (Levett) Blackborne, who was the daughter of Sir Richard Levett, Lord Mayor of London and the widow of merchant Abraham Blackborne, and her second husband Robert Thoroton of Screveton Hall, Nottinghamshire. (Robert Thoroton and his wife Mary became parties to a contentious lawsuit with the Blackborne family heirs — Thoroton vs. Blackborne — over an enormous estate left by William Hewer, longtime friend of diarist and Secretary of the Admiralty Samuel Pepys.)