The land was organized as part of the Town of Milwaukee in 1838, and much of the land was bought by land speculators, who cleared the old-growth forests for timber. The first permanent settler in the area was Cephas Buttles, who owned 160 acres in the Town of Milwaukee and built a cabin in present-day River Hills in 1843. In the mid-1840s, many families of Dutch settlers began arriving in the Town of Milwaukee. John Cappon was the first Dutch settler in Fox Point, building his cabin in 1846. By 1855, more than 20 Dutch families lived in Fox Point. The early settlers farmed the land with new techniques that allowed them to cultivate wheat, barley, rye, and corn in the clayey soil. In 1848, they built a one one-room schoolhouses and another two in 1852, including the Old Dutch Schoolhouse Site and Burial Ground, which is a Milwaukee County landmark. Two Dutch Reformed congregations formed in the community in its early years. The community's demographics changed later in the nineteenth century. One of Fox Point's first black residents was Calvin Reeves, a formerly enslaved man from Louisiana who moved to the community after the American Civil War with two Dutch-American soldiers returning home from the conflict. German immigrants also began moving to Fox Point in the 1860s, forming St. John's Lutheran Church in 1866.