Baldwin moved to Detroit, Michigan, where he established a wholesale business in boots and shoes in 1838. He was a member of the convention which organized the U.S. Republican Party in Jackson, Michigan in 1854. He was influential in organizing the sixth Episcopal parish in the northern outskirts of Detroit on December 27, 1858, and in building St. John's Episcopal Church for the parish. Baldwin donated the church lot and underwrote a large portion of the cost of building a chapel to seat 125 persons (completed November 1859) and paid for the construction of the rectory. He also contributed in large part to the building of the larger 1300 seat nave, completed in 1861. He was the Senior Warden from the parish's founding until his death.