Their lawyers told the Al-Sweady Inquiry "insufficient material" had been heard, but added they intended to pursue mistreatment claims.
The inquiry, set up in 2009, has cost more than Â£22m to date.
The Ministry of Defence said it was pleased the claims had been withdrawn.
The inquiry, which started hearing evidence last year, has sought to identify the events surrounding the deaths of allegedly more than 20 Iraqi men.
It has been concerned with the Battle of Danny Boy - named after a British checkpoint near the town of Majar al-Kabir in southern Iraq in May 2004, during the Iraq war.
Iraqi insurgents ambushed soldiers, leading to a three-hour firefight which included the use of bayonets.
Lawyers acting for several Iraqi clients claimed men taken from the battlefield were mistreated and unlawfully killed at Camp Abu Naji and Shaibah Logistics Base.
The MoD, British troops and their lawyers have vigorously denied the claims, saying those who died were killed on the battlefield.
A spokesman for the MoD said: "We have long said that there was no credible evidence for these allegations and are pleased that they have been withdrawn."
Speaking on the last full day of evidence, Public Interest Lawyers (PIL) for the families said they had arrived at their decision following "the conclusion of the military evidence and the current state of disclosure by the Ministry of Defence".
PIL's John Dickinson said in a statement: "From the outset, the families have had the simple objective of discovering the extent of any wrongdoing and, if so, how it came about and who was responsible.
"It is accepted that - on the material which has been disclosed to date - there is insufficient evidence to support a finding of unlawful killing in Camp Abu Naji."
Inquiry chairman Sir Thayne Forbes, who is due to release his final inquiry report in November, praised the lawyers for their admission.
He said: "You and your team are to be commended for the courage that you have displayed in making this statement at the stage that you did."
The inquiry has been sitting for 42 weeks and has heard evidence from a total of 281 witnesses.
It is named after one of the Iraqi men, 19-year-old Hamid al-Sweady, who was alleged to have been unlawfully killed while being held following the battle.
A public inquiry into whether UK soldiers unlawfully killed Iraqi civilians in custody in 2004 has heard their relatives no longer believe there is enough evidence to back the claims.