ryder had known it was going to be difficult to travel every weekend once the baby was born , but he had n't realized just how hard it would be to tear himself away , especially when his son was only four days old .