By the beginning of the 1980s he had a new rival, Chiyonofuji, who earned promotion to "ōzeki" and then "yokozuna" by defeating him in decisive matches in January and July 1981. In November 1981 Kitanoumi withdrew from a tournament for the first time. After that his record was patchy, with many absences. His 24th and final title came in May 1984, with a perfect 15–0 record. This was seen by many as a fitting end to a great career and he wanted to retire after that tournament, but was persuaded by the Sumo Association to carry on until the opening of the new Ryōgoku Kokugikan stadium in January 1985. Three days into the tournament, without winning a match, he announced his retirement. He had been ranked as a "yokozuna" on the "banzuke" in 63 tournaments, which remained the most in history until Hakuhō surpassed it in May 2018. During his career he had won 951 matches, the most in history at the time (he was overtaken by Ōshio in 1987). Of those victories, 804 came in the top division (a record broken by Chiyonofuji in 1991), and 670 of those came at the "yokozuna" rank.