When German troops invaded Belgium in 1940 the castle was confiscated and used as a local headquarters by the Nazis. Countess Jeanne moved back to the 'old castle' of Westerlo where she died on 1 July 1944, a few months before the liberation of Belgium. She left the castle to a monastic order of sisters. The building was used by the church as a home for retired priests until it was sold to the municipality of Westerlo in the 1970s. It has served as the Town Hall of Westerlo ever since.