It was during Patterson's reign as champion that he turned face, after a botched attempt by the Grand Wizard to "sell" Patterson's contract to "Captain" Lou Albano for $100,000; Albano's protégés, the Wild Samoans, attacked Patterson after he cut a promo insulting Albano. Patterson held the Intercontinental Heavyweight Championship until April 21, 1980 when he was defeated by Ken Patera in New York City, New York. The match ended in controversial fashion after Patterson placed his right leg on the ropes just before the three count was made. On May 4, 1981, Patterson's feud with Sgt. Slaughter culminated in an alley street fight match in Madison Square Garden. The match was voted Match of the Year by the "Wrestling Observer Newsletter".