as we approached the oldest part of town , where the beamed black-and-white upper floors of buildings leaned precariously over the cobbled pavement and fishermen 's cottages lined the streets , i could see one of the ferrymen deserting his post as he bent against the wind .