They were taken from Llanwenarth Baptist Chapel in Govilon overnight on Thursday, Gwent Police said.
The safes contained money and also marriage and funeral certificates, which were later found dumped close to the chapel.
Rev Peter Baines' wife, Frances, told BBC Wales she thought it looked like a "professional job".
Gwent Police appealed for witnesses to call 101.
"All the other items had been moved aside and nothing else was taken, just the two safes," Mrs Baines said.
Two safes have been stolen by thieves who broke into a chapel near Abergavenny.