Charles Seymour, 6th Duke of Somerset the pride of birth and rank amounted almost to a disease". Charles was the second son of Charles Seymour, 2nd Baron Seymour of Trowbridge (died 1665), of Marlborough Castle in Wiltshire, by his wife Elizabeth Alington (1635–1692). The 2nd baron was (in a junior line) a great-great-grandson of Edward Seymour, 1st Duke of Somerset (executed 1552), brother of Queen Jane Seymour, uncle of King Edward VI and Lord Protector of England. Charles was educated at Harrow School and Trinity College, Cambridge, where his portrait by Nathaniel Dance-Holland survives in the College's collection. In 1675, Charles's elder brother Francis Seymour, 5th
Edward Seymour, 8th Duke of Somerset and wife Elizabeth Somner, daughter of John Somner, of Seend. They had at least five children. In December 1740, his father died and Seymour inherited his manors in Wiltshire and Devon. On 11 September 1744, with the unexpected death of George Seymour, Viscount Beauchamp (1725–1744), the only son of Algernon Seymour, 7th Duke of Somerset, the likelihood emerged of Seymour succeeding his distant cousin as Duke of Somerset and Baron Seymour, as the 7th Duke was then aged sixty, and his wife was past child-bearing age. On 23 November 1750 the Duke died, and Seymour duly succeeded to his titles.