The area was also the site of the first Presbyterian Church in the colony, built in 1864 and situated in Mount Street. The congregation eventually outgrew the premises and had to move to a new church at Quarry Street, Ultimo, in 1883. In the 1870s, a small Methodist chapel was built in Harris Street on land owned by Charles Saunders. In the 1920s, it was converted to the Maybanke Free Kindergarten, named after Maybanke Anderson, a feminist and educationist. It was still used for that purpose as of 2018.