Brocket Hall early held variously of Hatfield Manor and the Bishop of Ely. Sir Matthew Lamb's son was Peniston Lamb, 1st Viscount Melbourne and he was often visited at Brocket Hall by the Prince Regent, who had a liaison with Lady Melbourne. The next owner was William Lamb, 2nd Viscount Melbourne, who was Queen Victoria's first Prime Minister from 1835–41, when she regularly visited. His wife, Lady Caroline, infamously had an affair with Lord Byron causing Lord Melbourne much embarrassment. For one of his birthdays she held a state banquet in the Saloon, at which she had herself served from a large
William Lamb, 2nd Viscount Melbourne replied: "Why, damn it all, such a position was never held by any Greek or Roman: and if it only lasts three months, it will be worth while to have been Prime Minister of England." "By God, that's true," Melbourne said, "I'll go!" Compromise was the key to many of Melbourne's actions. As an aristocrat, he had a vested interest in the "status quo". He was opposed to the Reform Act 1832 proposed by the Whigs, arguing that Catholic emancipation had not ended in the tranquility expected of it, but reluctantly agreed that it was necessary to forestall the threat