The discovery was made by canoeists close to Radyr Weir, Radyr, on Saturday shortly after 11:00 BST.
Officers spent about three hours at the scene as part of their investigation.
No further details have been released by police.
A body has been found in the River Taff in Cardiff, South Wales Police have said.