South Wales Police say the vehicle was spotted in the water at Blackweir in Bute Park on Tuesday morning.
Officers are trying to identify its owner, and say they are unable to confirm whether it is linked to a vehicle stolen from Cardiff council's Park department overnight.
There are no reports of anyone being injured.
Attempts are being made to haul out a submerged flat-bed truck from the River Taff in Cardiff.