The manor house, the site on which Firle Place now stands, was occupied from the early 14th century by the 'de Livet' (Levett) family, an ancient Sussex gentry family of Norman descent who owned the manor. The Levett family would later include founders of Sussex's iron industry, royal courtiers, knights, rectors, an Oxford University dean, a prominent early physician and medical educator, and even a lord mayor of London. An ancient bronze seal found in the 1800s near Eastbourne, now in the collection of the Lewes Castle Museum, shows the coat-of-arms of John Livet and is believed to have belonged to the first member of the family named lord of Firle in 1316. On the bankruptcy of lord of the manor Thomas Levett in 1440, the ownership passed to Bartholomew Bolney, whose daughter married William Gage in 1472. Following the death of Bolney in 1476 without a male heir, the seat of Firle Place was passed to William Gage and has remained the seat of the Viscount Gage ever since.