and , although it had been somewhat amusing to see him sit back down to silently scowl - sulk - taking isa 's sleeping form back from where he had passed her off when he had tried to intimidate me with his height , i still worried i had n't made the right choice leaving her with him .