Moore was injured in 1962, a knee injury, and lost his starting job in 1963 from the knee injury. In 1964, he had one of his best statistical seasons when he scored 20 touchdowns, helping to lead the Colts to a 12–2 regular-season record and a trip to the NFL Championship Game for the third time in seven seasons. He was voted by his fellow players as the Most Valuable Player, a remarkable award considering he played most of the season with complications from appendicitis, and gave credence to the claim that he was one of the toughest players in the NFL. He also won Comeback Player of the Year Award in 1964.