The first decapitated swan was found at The Quay on Saturday. The second was found in the same area on Wednesday.
The RSPB charity said it "beggars belief" that anyone could harm the "beautiful and much loved" birds.
Killing swans, which are a protected species, is a crime under the Wildlife and Countryside Act. People who do so could face a Â£5,000 fine per offence or up to six months in prison.
More on the swan attacks, plus more Devon and Cornwall news
Thomas Prideaux found one of the birds while on his way to work.
He said: "It had a what looked like a pellet hole in its side and its head was clean cut off. There were feathers everywhere, but the head wasn't to be seen anywhere."
Exeter City Council confirmed the "first decapitated swan was brought to the canal office last Saturday". It was reported to Defra on Monday.
Devon and Cornwall Police said they believed the death of the second bird "appeared to have been a deliberate act".
An RSPCA spokesperson said: "It is very distressing to think that someone has carried out such horrific acts towards these defenceless swans.
"We would appeal to people's consciences to report any information they have about the deaths to the police."
The force has closed its investigation due to lack of witnesses or CCTV evidence.
Two swans have been found beheaded in Exeter.