of course he never married again and in the end he got shunted off into the ground nuts scheme , and when that was a failure he retired and went to live in nigeria - back to the only people in the world who had shown him any kindness - back to where it had all started from .