In the UK, common scoters breed at only a few locations in the Flow Country of Caithness and Sutherland and lochs in the hills and glens near Inverness.
A key cause is now thought to be rising numbers of trout which eat the ducks' main food source, freshwater insects.
RSPB Scotland and others have raised concerns the bird could become extinct locally because of poor breeding.
The charity suspects declining angling on the lochs has helped boost brown trout populations.
Dr Mark Hancock, from the RSPB's Centre for Conservation Science said: "Of all the lochs we investigated during this work, scoters bred most often at those with the shallowest water and the most large, freshwater invertebrates.
"It soon became clear that there were more insects where there were fewer brown trout, so it looks like scoters are being limited by a lack of food in places where the fish are eating it all."
"We're now using these results to design new ways of helping scoters.
"For example, in areas of the north Highlands where angling activity has dropped off and fish numbers have increased, more trout angling is potentially one way to boost freshwater insect life."
The common scoter project was supported by Scottish Natural Heritage (SNH), the Wildfowl and Wetlands Trust (WWT) and The Conservation Volunteers (TCV).
Dr Andy Douse of SNH and co-author of the study, said: "Scotland is the only part of the UK to have breeding scoters, many of which nest in legally-protected nature conservation sites.
"This study highlights promising management options for restoring populations of this declining species."
Another of the co-authors, Hannah Robson, of WWT, added: "Scoters are amazing birds: an arctic species finding a haven in remote Scottish lochs.
"This research points to the ways in which we might be able to save them as a fascinating part of our Scottish wildlife heritage."
Conservationists believe they have identified the cause of a decline in numbers of a rare duck.