from where they watched and waited they could see down onto the ramparts of the small fort , and were surprised to see that no one was obviously manning the three cannon that were visible atop the walls and pointing towards the harbor .