In 1974, Fisher was elected to Congress from Virginia's 10th congressional district. He served for three terms until his defeat at the hands of Republican Frank Wolf in November 1980. He went on to establish the Economic Policy Department at The Wilderness Society, a U.S. non-governmental organization, bringing a first-of-its-kind professional scientific focus to the wildland conservation community. Afterward, Fisher was appointed Virginia Secretary of Human Resources in 1982 and then became an economics professor at George Mason University in 1986.