Another crew member from the vessel, which has been named locally as the Stornoway-registered Louisa, is missing.
One man was recovered from the water shortly after the alarm was raised at 03:45 and taken by helicopter to hospital.
The coastguard said at 14:45 that the search had been scaled back.
The operation was in an area of sea close to the island of Mingulay, where a signal from a distress beacon was located.
Coastguard helicopters from Stornoway and Prestwick took part, along with the Barra RNLI lifeboat.
Mark Rodaway of UK Coastguard said: "Despite an intensive search including the helicopters, lifeboat and other fishing vessels in the area, we have been unable to locate the missing fisherman. Our thoughts are with all those involved"
Ch Insp Alastair Garrow, of Police Scotland, said: "At this time we can confirm that the bodies of two men have been recovered. A third man was rescued and was taken to hospital at Stornoway. He is not seriously injured.
"A fourth man was on the boat and is still missing. The next of kin of all the men have been informed.
"An investigation will be carried out in parallel with the police and the Marine Accident Investigation Branch (MAIB) and a report will be submitted to the Procurator Fiscal."
He added: "This has been a tragic incident which will impact on the local community. Our thoughts are with the families affected."
Two people are confirmed to have died after a fishing boat went down off the Western Isles.