The gruesome sight has haunted residents like Ayaan Mohamed, who lives near the graveyard.
"Seven families including mine have fled the neighbourhood," she told the BBC.
With houses partially submerged and human remains floating nearby, they fear an outbreak of disease, she says.
Some of the bodies were recognisable, further traumatising people - and as the water has subsided unearthed bones have also been found.
"The floods revealed the remains of a highly esteemed Muslim cleric. He was laid to rest 18 years ago," Ms Mohamed said.
"His students and other clerics attempted to gather the remains," she said, but they were unable to do so.
These are scenes the city has never witnessed before.
At least 32 people have been killed across the country and the UN warns that more than 1.6 million people could be affected in the devastating floods which follow years of drought.
The situation in Galkayo is not as a severe as in the Gedo region of southern Somalia, where the River Juba has burst its banks and swept away a key bridge in the city of Bardere.
It was overwhelmed by floodwaters and collapsed on Saturday.
Mohamed Abdirahin says almost all the city's residents have been forced to move to the outskirts.
The 70-year-old was a prosperous merchant and farmer until last week.
"I left everything behind," he told the BBC.
"I salvaged what I could with my bare hands from my shop - the rest was swept away by the water. It's like my life ended like that," Mr Abdirahin says.
He is finding it hard to come to terms with how swiftly his life was transformed as floodwaters first inundated his shop, then brought down his home.
Mr Abdirahin also had a farm just outside the city, where he went to check on his mature fruit trees after his family were forced to leave their compound in the city.
He had been hoping for a bountiful harvest at the end of January, but that dream has been swept away along with the fruit.
"The 10m- (32ft-) high mango trees had almost sunk when we were leaving, I couldn't see anything standing on my land," Mr Abdirahin said.
One part of the city, which is on higher ground, has been spared, but it has become inaccessible by road without the bridge.
It is a vital lifeline for all those that have lost their homes as the district has food stores and a health care centre.
People have to use small boats to reach there.
"We wait more than four hours for those boats to go and get for us something to eat," Mr Abdirahin said.
Somalia's fragile, UN-backed government has been trying to provide emergency relief, but it says it cannot cover all the affected areas.
"For the last five days, we have supplied many materials for those in need - the current situation has overwhelmed the government capabilities," Somalia's Deputy Prime Minister Salah Jama told the BBC.
Another problem is that some areas are not in the hands of the government and are controlled by the al-Qaeda-linked group al-Shabab.
The Somali Disaster Management Agency (Sodma) says it has been receiving regular calls from people living in al-Shabab territory, but it is not able to help as it would be too dangerous for officials to venture there.
In some cities people whose houses are still standing have been told to take in those who have been made homeless - and some communities have been clubbing together to cook meals for one another.
When the long-awaited rainy season arrives in Somalia - it is usually a time for celebration, symbolising prosperity to come, especially when the country has been increasingly experiencing droughts.
People often venture outdoors to explore the ponds and watering holes that fill up and sometimes even go swimming.
The niece and daughter of Anab Deerow, who lives in El-Wak west of Bardere on the border with Kenya, went out to have fun after a heavy downpour.
Ms Deerow said the two girls had been so excited as they had just finished their exams and it was the first day of their holidays before starting senior school next year.
The cousins had been playing outside for a few hours, taking selfies and had just clambered down to see a pool in a small valley outside the town when a torrent of water swept them away.
Ms Deerow's eight-year-old daughter witnessed it and came screaming home to tell her. The girls' lifeless bodies were found later following a long search after the water had subsided.
"They had returned from school a day before they died. I'm at a loss for words. It seems the angel of death called them out," Ms Deerow told the BBC.
The heavy rainfall and floods are expected to continue until the end of November, according to Sodma.
As well as those who have had to leave their homes, 1.5 million hectares (3.7 million acres) of farmland could potentially be destroyed, the UN warns.
The thought of more rain falling is terrifying for many people already struggling to find food and clean water.
Mr Abdirahin and his family of 12 now find themselves sleeping in the dense bush land outside Bardere, where they have been for more than a week.
"We are totally lost in the bushes," he said, becoming overwhelmed with emotion.
"We are just alive for no reason. We are terrified. Just waiting for help from God."
More on Somalia:
Following Somalia's worst floods in a century, gushing waters have swept through a cemetery in the central city of Galkayo, leaving bodies floating through the streets.