there was a quarry where tons of what appears to be a sandy stone were blasted out of the quarry wall and hauled on carts by some convicts up to the rest of us , who would then use our hammers to break the bigger pieces into littler pieces that would be tamped into the space behind the seawall .