but it was n't his hands on me that brought him to grief , and the simple truth is that , no matter what people on the island might have thought at the time , he never hit me a single lick during the last three years of our marriage .