In January 1970, Hunter was chosen by then-President Richard Nixon to become the chairman of the Federal National Mortgage Association, the nation's largest provider of housing finance. In 1978, tension started to rise between Hunter and Secretary of Housing and Urban Development Patricia Roberts Harris. Harris specifically felt that the organization was too concerned about making money and "too unconcerned with stimulating mortgage lending for low-income housing in the cities." Hunter's resignation was being called for by the Carter White House and by the directors of the company's board. In November 1977, a vote for Hunter's resignation failed by an 8-6 margin.