The Lennox sisters were daughters of Charles Lennox, 2nd Duke of Richmond, and Sarah Cadogan. Charles Lennox was the grandson of Charles II of England through the King's relationship with Louise de Kérouaille, Duchess of Portsmouth. In 1744, Lady Caroline eloped with Henry Fox, a politician who was eighteen years her senior. Though her parents disapproved of the marriage, it proved a very happy one. The couple had four sons, including the great Whig politician, Charles James Fox and the general Henry Edward Fox. Their home, Holland House, Kensington, was a social and political gathering place.