No sleepovers! The children should only see the grandfather when the parents are present. The grandmother should be welcome to come to their house and look after the children but grandpa is not allowed to interact with the children unsupervised by the parents. The grandmother stood by and watched it happen and did not intervene with her own children. These are their children and they want them raised the way they want.