Apparently he could not escape that part of his past, and there was no reason to deny it.
He could not get rid of that time in his past, and there wasn't a reason to deny it.
He was able to shake off his past and start anew.