It was from Minneapolis that Johnson was recruited, along with his defensive partner Taffy Abel, to play for the newly formed New York Rangers. Offered the position at the age of 28, Johnson insisted on a three-year contract as he believed it would be the only one he would get. He made his NHL debut in the Rangers opening game on November 16, 1926 in a 1–0 victory over the Montreal Maroons. A rugged and physical defenceman, Johnson appeared in only 27 of the Rangers' 44 games as he suffered a broken collar bone early in the season, but still finished second on the Rangers with 66 penalties in minutes. In 1927–28, his total of 146 penalty minutes led the team and was second in the NHL to Eddie Shore's 165. He added a career high 10 goals, and helped the Rangers win their first Stanley Cup. Additionally, he was voted by the fans as the most valuable player of either New York team.