Frances Villiers, Countess of Jersey his "wife", Maria Fitzherbert, with whom he had undergone a form of marriage in a clandestine Church of England ceremony that all parties to it knew was invalid under the Royal Marriages Act 1772. Having encouraged the Prince of Wales to marry his first cousin, Caroline of Brunswick in 1794, Lady Jersey nevertheless set out to make Caroline's life difficult. However, the now Princess of Wales (Caroline) had very little regard for George IV, nor he for her, and after the birth of their child, they lived apart during their twenty-five year marriage, leaving a void Frances and other mistresses,
Villiers family sons to inherit that title. Sir Edward Villiers's eldest son, William († 1643), thus succeeded as second Viscount Grandison in 1630. He was the father of Barbara Villiers († 1709), one of the mistresses of King Charles II, by whom she had five children, and who was created Duchess of Cleveland in 1670. Sir Edward Villiers's second and third sons, John († c.1661) and George († 1699), succeeded as 3rd and 4th Viscounts Grandison, while the fourth son, Sir Edward Villiers († 1689), was father of Edward Villiers († 1711), who was created both "Baron Villiers" and "Viscount Villiers" in