Brocket Hall early held variously of Hatfield Manor and the Bishop of Ely. Sir Matthew Lamb's son was Peniston Lamb, 1st Viscount Melbourne and he was often visited at Brocket Hall by the Prince Regent, who had a liaison with Lady Melbourne. The next owner was William Lamb, 2nd Viscount Melbourne, who was Queen Victoria's first Prime Minister from 1835–41, when she regularly visited. His wife, Lady Caroline, infamously had an affair with Lord Byron causing Lord Melbourne much embarrassment. For one of his birthdays she held a state banquet in the Saloon, at which she had herself served from a large
William Fortescue, 2nd Viscount Clermont William Fortescue, 2nd Viscount Clermont William Charles Fortescue, 2nd Viscount Clermont (12 October 1764 – 24 June 1829), was an Irish politician. Fortescue was the son of James Fortescue by his wife Mary Henrietta Hunter, a daughter of Thomas Orby Hunter, of Crowland Abbey, Lincolnshire. His uncle was William Fortescue, 1st Earl of Clermont, 1st Viscount Clermont. He served in the British Army, reaching the rank of lieutenant. In 1796 he was returned to the Irish House of Commons for County Louth (succeeding his brother Thomas James Fortescue), a seat he held until 1800, when the Irish Parliament was abolished