Police said the man was pulled from the water by five members of the public but later died in hospital.
The incident happened at about 17:00 on Monday at the Falshaw Bridge, near Arboretum Avenue in Stockbridge. There are no suspicious circumstances.
Police said they had spoken to the four women who were involved in attempting to save the man.
But they appealed for a man who had also helped to contact them.
Sgt Donna Mackay said:  "Our condolences are with the friends and family of the victim who passed away, despite the valiant efforts of these five good Samaritans.
"We are eager to trace the final male witness as part of our enquiries.  We would ask this man, or anyone who may know who he is, to contact police as soon as possible."
A 58-year-old man has died after falling into the Water of Leith in Edinburgh.