Police are treating the victim's death as unexplained, following the discovery near East Cowes just before 09:00 GMT.
A Hampshire Constabulary spokesman said investigations were being carried out to determine the man's identity.
He added that "the death is being treated as unexplained at this time" and asked anyone with information to call the force on 101.
An investigation has begun after man's body was pulled from the water off the Isle of Wight.