In 1841 the census said there were about 259 people living in Williamstown. It is believed that there were many more. There were three hotels in Williamstown by 1841 and most of the men worked at the port. Being a busy port, there were numerous lodging houses and a constantly changing population. The years 1842 and 1843 there was an economic recession and Williamstown's population went down.