grandfather hines had insisted he did it and truth was he enjoyed working out , he could remember his mother saying that her father could have been the heavyweight champ , but was not allowed to pursue it .