Police and HM Coastguard were called after a report of a body in the water at Hawes Pier at about 10:20 on Thursday.
The man was pronounced dead at the scene. His death is not being treated as suspicious.
Officers said he was not carrying any identification and inquiries were under way to find out the man's name.
The man was white, about 5ft 10ins tall with blackish-grey receding hair, in his late 40s and with a medium build.
He was wearing a black button-front jacket, blue hooded top with full zip and white toggles, blue denim Levi jeans, black hat, black scarf and navy gloves.
Det Sgt Kevin Tait, of Corstorphine Police Station, said: "Our investigation is continuing to establish this man's identity and we would ask anyone who can assist with our ongoing inquiries to contact police on 101."
Detectives are trying to identify the body of man who was found on the shoreline at South Queensferry.