Brocket Hall early held variously of Hatfield Manor and the Bishop of Ely. Sir Matthew Lamb's son was Peniston Lamb, 1st Viscount Melbourne and he was often visited at Brocket Hall by the Prince Regent, who had a liaison with Lady Melbourne. The next owner was William Lamb, 2nd Viscount Melbourne, who was Queen Victoria's first Prime Minister from 1835–41, when she regularly visited. His wife, Lady Caroline, infamously had an affair with Lord Byron causing Lord Melbourne much embarrassment. For one of his birthdays she held a state banquet in the Saloon, at which she had herself served from a large
Peniston Lamb, 1st Viscount Melbourne and inherited Melbourne Hall in Derbyshire. He married Elizabeth Milbanke (1751–1818), daughter of Sir Ralph Milbanke, 5th Baronet, on the 13th April 1769. She was a young woman of great beauty, intelligence and strong character, who quickly came to dominate her husband completely, and steered them into the centre of polite society. In 1770 he began, as Melbourne House, what is now The Albany in London. At the 1768 general election Lamb was returned unopposed as Member of Parliament for Ludgershall. In 1770 he was raised to the Peerage of Ireland as Lord Melbourne, Baron of Kilmore, in the County