William Berkeley, 1st Earl FitzHardinge refused to claim his right to the earldom. In 1831 William Berkeley was raised to the peerage as Baron Segrave, of Berkeley Castle in the County of Gloucester. Berkeley was returned to parliament as one of two representatives for Gloucestershire in 1810 (succeeding his uncle Sir George Cranfield-Berkeley), a seat he only held until 1811. In 1836 he was appointed Lord-Lieutenant of Gloucestershire, a post he retained until his death. In 1841 he was further honoured when he was made Earl FitzHardinge. Berkeley never married. He had several mistresses, and in 1821 John Waterhouse succeeded in an action for "criminal
Kenilworth Castle of the Midlands, in an area that held several nobles who still supported Edward and were believed to be trying to rescue him. Henry's loyalty was also coming under question. In due course, Isabella and Mortimer had Edward moved by night to Berkeley Castle, where he died shortly afterwards. Isabella continued to use Kenilworth as a royal castle until her fall from power in 1330. Henry of Grosmont, the Duke of Lancaster, inherited the castle from his father in 1345 and remodelled the great hall with a grander interior and roof. On his death Blanche of Lancaster inherited the castle.