At the age of sixteen, James played with the Winnipeg Monarchs junior hockey team in the 1951 Memorial Cup (a loss). The Toronto Maple Leafs, who owned James' professional hockey rights, decided to move him to Toronto to play for the Toronto Marlboros, their top junior team. James would win the 1955 Memorial Cup playing with the Marlboros – only a few months after winning the CFL's Most Outstanding Canadian award. A few days after the Memorial Cup win, James also played his first NHL game with the Maple Leafs – ending a tremendous series of multi-sport and multi-league achievements within a five-month period.