A prominent structure in the area's history is the Shafer Grist Mill. Originally built by Casper Shafer in 1741, it was moved approximately a half mile to its present location in 1764. It was rebuilt in 1844 and powered by water from the Paulins Kill. Casper's son, Nathan Armstrong Shafer lived in Whitehall Manor, which he built near what is now the center of the village. The gristmill operated commercially until 1955, making it one of the oldest, continuously operating, water-powered gristmills in the State of New Jersey. Aline Murray Kilmer, the widow of poet Joyce Kilmer, lived the last decade of her life at Whitehall, passing away there on October 1, 1941.