granby did not answer again , until later , when they had come to laurence 's quarters and might shut the door behind them ; in that privacy he bluntly said , `` then they ca n't stop us going , either . ''