In the 16th century, the Kamenny Monastery did not develop as quickly as the two last-mentioned abbeys, because of the limited territory that the tiny and remote island could afford. In the 1540s, the monks constructed a church-belltower of curious architecture and a refectory, after which the monastery gradually declined to obscurity. It was remembered in Moscow primarily as a place of exile, where the famous Old Believer, Ivan Neronov, was deported. On 24 July 1774 one of the exiles set the monastery on fire, whereupon the monks were transferred for 26 years to Vologda. When they finally returned to the island in 1801, the cathedral was restored with five domes instead of one. The only post-medieval buildings on the island were an inn and two lighthouses, built for the needs of the monastery in the 1870s.