Hatherton resided at Teddesley, where he established a free agricultural college and farmed successfully. He strongly promoted education in the area, paying for a National School in Penkridge and another at Levedale, and for clothing for some of the school children. However, his lifetime saw a decisive shift in the family's interests. As heir to both the Walhouse family fortune and the Littleton estates, he owned great estates around Penkridge and mineral holdings and much residential property in the Cannock and Walsall areas. He owned coal mines at Great Wyrley, Bloxwich and Walsall; limestone quarries and brickyards in Walsall that were used to build much of the town; hundreds of residential and commercial properties; gravel and sand pits, stone quarries in many places. Unlike Penkridge, Cannock and Walsall were boom towns of the Victorian era, powered by the most modern and profitable industries of the age. The Littletons played a leading part in this phase of the Industrial Revolution and made large profits from it, and this tilted their attention increasingly away from their landed estates.