Spencer Mountain was settled in 1772 by Zachariah Spencer, a local Tory. In 1874, J. Harvey Wilson II built a water-powered textile mill (Wilson & Moore Cotton Mill) at the base of Spencer Mountain. W. Thomas Love and John C. Rankin purchased the mill in 1895 and changed its name to Spencer Mountain Mills. A predecessor of Duke Power purchased the mill and the hydroelectric plant that powered it in 1926. The mill was purchased by Pharr Yarns in 1957 and was shut down in 1999. Spencer Mountain was incorporated for a brief period of time between 1895 and 1909 and again in 1963.