Andrews served only in the 89th Congress from January 3, 1965, to January 3, 1967. He and other Alabama members opposed the Voting Rights Act of 1965, which passed after the landmark African American-led March from Selma to the state capital at Montgomery. In 1966, Andrews was defeated for reelection by about the same margin that he had won in 1964. He was unseated by the Democratic State Senator Bill Nichols. Nichols received 54,515 votes (58.7 percent) to Andrews' 38,402 (41.3 percent).