and then , after a short rest in the bottom of the boat , the work went on until four o'clock , when a small chill breeze came from nowhere and got between them and the warmth of the sun and it was time to make the long row home .