Brocket Hall early held variously of Hatfield Manor and the Bishop of Ely. Sir Matthew Lamb's son was Peniston Lamb, 1st Viscount Melbourne and he was often visited at Brocket Hall by the Prince Regent, who had a liaison with Lady Melbourne. The next owner was William Lamb, 2nd Viscount Melbourne, who was Queen Victoria's first Prime Minister from 1835–41, when she regularly visited. His wife, Lady Caroline, infamously had an affair with Lord Byron causing Lord Melbourne much embarrassment. For one of his birthdays she held a state banquet in the Saloon, at which she had herself served from a large
Lady Caroline Lamb of making poetesses of her pupils". In her early adult years, Lady Caroline not only wrote prose and poetry, but also took to sketching portraiture. These courtly skills stood her in good stead. She spoke French and Italian fluently, was skilled at Greek and Latin, and also enjoyed music and drama. In June 1805, at the age of nineteen, Lady Caroline Ponsonby married the Hon. William Lamb, an up-and-coming politician, and heir to the 1st Viscount Melbourne. Although their meeting had been shrewdly orchestrated by William Lamb's mother, theirs was a love match. The couple had become "mutually captivated" during