The area was first developed for agriculture, and planters used enslaved African Americans for labor before the Civil War. Early crops included potatoes, cotton, corn, rice, and sugar cane, and commercial enterprises included cattle, molasses, and sugar. At different times, the community had a cottonseed oil mill, a sugar cane factory, gristmills, cotton gins, a milk processing plant and dairy, an ice plant, and numerous other industries. Oil and sulfur production in the outlying areas contributes to the town's economy. The population of Wharton was about 200 in the early 1880s.