In 1962, the 6 foot 5 inch actor was socialising with fellow expatriates in a London club when they were joined by an Australian who acted as doorman, and unbeknownst to Rafferty, was a professional wrestler. Claiming he was being ignored after helping them get in the doorman was so argumentative that Rafferty was provoked into accepting a challenge to 'step outside'. In the severe beating that followed he sustained deep grazing across his face and suffered a myocardial infarction (he had not been aware of having a heart condition until the incident) costing him the chance at roles in two major film productions.