The courthouse (see below) was built between 1833 and 1838. The gaol (see below) was built from 1835 by convict labour and opened in 1839. The "Surveyor General Inn" was built in 1834. It has been continuously licensed since 1839 and its claim to being the earliest hotel rests on its continual licence and being in the original building. Berrima prospered as being at a point on the Old Hume Highway, and there were fourteen hotels in or near the town in the 1840s. However, because the construction of the railway bypassed the town, the population decreased – no new houses were built for a hundred years. In 1896, Sir Henry Parkes, premier of New South Wales, planted an oak tree near the post office.