As the Whig Party disintegrated in 1852, Stuart declined to become its candidate in the U.S. Senate. Instead, he aligned himself with the nativist Know Nothing Party, which some criticized as secretly working against slavery. When Virginia's Democratic governor Henry A. Wise criticized the Know Nothings, Stuart (as "Madison") published twelve long letters on the "American Question" in the "Richmond Whig and Public Advertiser" and later as a combined pamphlet. Neither endorsing nor opposing slavery, Stuart praised American Party's proposals to deny some immigrants rights accorded native-born Americans.