After resigning as Florida's head football coach, Graves remained the university's athletic director until 1979, a position he had also held since becoming the football coach in 1960. His remaining tenure as athletic director was notable for the University of Florida embracing the challenges and opportunities in women's college sports presented by Title IX. After he was approached by Professor Dr. Ruth H. Alexander and other female coaches to fund women's athletics, their proposal was brought to the Faculty Committee on Intercollegiate Athletics who approved their budgeting plan for the 1972–73 season. Under Graves' guidance, Florida's women's sports program began its climb to become one of the top ten women's programs in Division I sports. After he retired as athletic director in 1979, he worked for another year as a special assistant for fundraising in the office of the university president.