Evans retired from umpiring following the 1927 season to become the general manager of the Cleveland Indians, earning a substantial annual salary of $30,000. Baseball historian Bill James observed that Evans was the first front-office executive of a major league team to be officially called a "general manager". In this capacity, Evans was credited with taking the Indians from a second division to a first division team. He served as general manager for the next eight years, until budget cuts forced him out in 1935. Rumors circulated that Evans's decision to leave the Indians was also motivated by a disagreement with the Indians' manager, Walter Johnson, over the suspension of third baseman Willie Kamm and the release of catcher Glenn Myatt. Johnson allegedly accused Evans of "disloyalty", while Evans reportedly replied that he refused to be a "yes man". Evans soon found work as chief scout and head of the Boston Red Sox farm system, but left on October 8, 1940, after the team sold Pee Wee Reese to the Brooklyn Dodgers over his objections.