Buccleuch married, firstly, Lady Jane Douglas, daughter of James Douglas, 2nd Duke of Queensberry, and Mary Boyle, on 5April 1720 in Earl of Rochester's House, Privy Gardens, Whitehall. They had two sons (Francis and Charles) and three daughters. Lady Jane died in 1729 and is buried at Dalkeith Castle. He married, secondly, Alice Powell, daughter of Joseph Powell, on 4September 1744 in St. George's Chapel, Mayfair, London, England. Both his sons predeceased him, so he was succeeded by his grandson Henry Scott, 3rd Duke of Buccleuch, the son of Francis. According to his will, he had six other children by Sarah Atkinson. He also appears to have had a son and three daughters by Elizabeth Jenkins.