The fortunes of the town and the Littletons remained intertwined. Sir Edward Littleton, the fourth baronet, bought Penkridge manor from the Earl of Warwick in 1749, completing his family's dominance of the area. Soon after he built Teddesley Hall, a much more impressive seat for the family. He survived until 1812, although, his wife died childless in 1781. He adopted his great-nephew, Edward Walhouse, as his heir. Walhouse took the name Littleton and took over the Littleton estates, although not the Littleton baronetcy. He achieved far greater eminence as a politician than any other member of the family, serving as Chief Secretary for Ireland under the Whig Prime Minister Grey in 1833–35. He was elevated to the peerage as Baron Hatherton, a title which remains with the head of the Littleton family to the present, and became an active member of the House of Lords.