William Berkeley, 1st Earl FitzHardinge refused to claim his right to the earldom. In 1831 William Berkeley was raised to the peerage as Baron Segrave, of Berkeley Castle in the County of Gloucester. Berkeley was returned to parliament as one of two representatives for Gloucestershire in 1810 (succeeding his uncle Sir George Cranfield-Berkeley), a seat he only held until 1811. In 1836 he was appointed Lord-Lieutenant of Gloucestershire, a post he retained until his death. In 1841 he was further honoured when he was made Earl FitzHardinge. Berkeley never married. He had several mistresses, and in 1821 John Waterhouse succeeded in an action for "criminal
Berkeley Castle around 1067 by William FitzOsbern shortly after the Conquest. This was subsequently held by three generations of the first Berkeley family, all called Roger de Berkeley, and rebuilt by them in the first half of the 12th century. The last Roger de Berkeley was dispossessed in 1152 for withholding his allegiance from the House of Plantagenet during the conflict of The Anarchy, and the feudal barony of Berkeley was then granted to Robert Fitzharding, a wealthy burgess of Bristol and supporter of the Plantagenets. He was the founder of the Berkeley family which still holds the castle. In 1153–54, Fitzharding