Crutchfield attended the first session of the pro-Union East Tennessee Convention in Knoxville in late May 1861, and campaigned against secession in the Chattanooga area. He remained in Chattanooga for most of the Civil War. Amid the crackdown during the aftermath of the East Tennessee bridge burnings in November 1861, Crutchfield was arrested by Confederate authorities, but managed to escape. During the war, his family sold the Crutchfield House for $65,000 in Confederate money, and used the proceeds to purchase land and tobacco. Crutchfield would later sell the tobacco at a profit to Union soldiers.