He lost a contested seat that year for United States Representative from South Carolina's 2nd congressional district to Democrat Albert W. Watson, who had the support of powerful senator Strom Thurmond. Watson shifted to the Republican Party in 1965 and ran unsuccessfully for governor in 1970. That year Spence won the congressional seat, and was re-elected for fourteen terms after this. He became ranking Republican on the House Armed Services Committee in 1993 and chairman in 1995. Spence died in office from cerebral thrombosis in Washington, D.C. in 2001.