Washburn was a native of Lynn, Massachusetts, and was raised in Ludlow, Vermont. he graduated from Dartmouth College in 1835, and taught school while studying law. He was admitted to the bar in 1838, and practiced law, first in Ludlow and then in Woodstock, while also becoming active in politics as a Whig. In addition to campaigning for Whigs including William Henry Harrison, Washburn held elected and appointed offices, including Reporter of Decisions for the Vermont Supreme Court and member of the Vermont House of Representatives.