In 1852, Madison's brother, Eston, moved with his family away from Ohio (and his brother) to Madison, Wisconsin, to get further from possible danger from slave catchers following passage of the Fugitive Slave Act of 1850. Slave catchers were known to kidnap free black people and sell them into slavery, as demand and prices were high in the Deep South. In Wisconsin, Eston and his family all took the surname Jefferson and entered the white community. They lived according to their appearance and mostly white ancestry. Their oldest son John Wayles Jefferson served as a regular Union officer in the American Civil War, and was promoted to colonel. Their son Beverly also served in the Union Army and married a white woman. Their daughter Anna married a white man. All of Eston's descendants identified as white.