he took his hand away and looked at his sister and brother , as if half-expecting them to say something , but they were either too immersed in their own misery to speak or too afraid of sharing his fate if they alienated her .