Poultney was first settled by Thomas Ashley and Ebenezer Allen. Ashley married Zeruiah Richards, and Allen married Lydia Richards, both daughters of Zebulon Richards of Windham County, Connecticut. Ashley and Allen established themselves in a cabin near the Poultney River on February 15, 1771. Allen brought his family with him, and a son born to his wife Lydia in 1772 was the first white child born in Poultney. Ashley built a shanty and planted corn before bringing his family to Poultney—he returned with his family in the fall of 1771. Six of Ashley’s brothers followed him to Poultney during the next two years, and several members of the Richards family were also early settlers.