The choir and crypt are in the Romanesque style and stand out by their simplicity. The crypt contains the tombs of Saint Ursmarus and Saint Erminus, as well as tombstones from other abbots.
The church of Saint Ursmarus is one of the oldest churches in Belgium. The oldest parts date from Carolingian or Ottonian times.