After Prussia and the Russian Empire made the First Partition of Poland in 1772, the town was annexed by the Kingdom of Prussia and in 1773 it became part of the newly established province of West Prussia. At that time, the officials used the rather neglected castle as a poorhouse and barracks for the Prussian Army. The last Jesuits left the castle in 1780. In 1794 David Gilly, a Prussian architect and head of the Oberbaudepartement, made a structural survey of the castle, to decide about its future use or demolition. Gilly's son, Friedrich Gilly, produced several engravings of the castle and its architecture, which he exhibited in Berlin and had published by Friedrich Frick from 1799 to 1803. These engravings led the Prussian public to "rediscover" the castle and the history of the Teutonic Knights.