The Reverend Rufus Noy called in the animals to clear plots at St Peter's Church, Blaenavon, after volunteers fell behind with their chores.
Mr Noy said maintaining the site was not the council's responsibility and volunteers had struggled.
Farmer Nick Prewett-King said his South American goats were happy to help.
Mr Noy, 53, said: "We have struggled for too many years in fighting the worsening state of the graveyard and although a lot of people won't know this, bringing animals to eat the grass has happened before, as I believe we had horses doing a similar job in the early 1980s."
Mr Prewett-King, 58, said the overgrown plants, grass and brambles were perfect for the goats because the area is pesticide-free and they "go mad for it".
"Because of the climate they are used to, the weather has to be just right for them to be happy to be outside and eat so we had to wait a day but it's excellent for them and really supplements their diet around the oats, wheat and berries we give them," he said.
Mr Noy said he hoped the goats would continue to help out, adding: "It's a great idea as we both benefit from it, anything that can help with the overgrowth is good as it's hard work."
St Peter's is the earliest industrial church in the area and was built in 1804 by Samuel Hopkins and Thomas Hill, who owned the town's ironworks.
A herd of goats has been appointed to work the graveyard shift at a Blaenau Gwent church where weeds have taken over the cemetery.