A recent report revealed that European companies were exploiting weak regulations in West Africa to export fuels with high levels of sulphur.
Nigeria, Benin, Togo, Ghana, and Cote d'Ivoire agreed on the import ban.
The UN says the move will help more than 250 million people breathe safer and cleaner air.
The sulphur particles emitted by a diesel engine are considered to be a major contributor to air pollution and are ranked by the World Health Organization (WHO) as one of the top global health risks.
It is associated with heart disease, lung cancer and respiratory problems.
In a statement, the UN environment programme says the West African group, in addition to banning the import of dirty fuels, has also agreed to upgrade the operations of their national refineries.
The upgrade, which will concern both public and privately owned refineries, is meant to boost standards in the oil produced in the five countries.
The report into Europe oil exports released in September particularly criticised the Swiss for their links to the African trade in diesel that has toxin levels illegal in Europe.
The study was by the campaign group, Public Eye, which said retailers were exploiting weak regulatory standards.
Erik Solheim, the head of UN Environment Programme, hailed the import ban.
"West Africa is sending a strong message that it is no longer accepting dirty fuels from Europe," Mr Solheim said.
"Their decision to set strict new standards for cleaner, safer fuels and advanced vehicle emission standards shows they are placing the health of their people first," he added
For a long time African countries relied on colonial-era standards, which have only been revised in recent years.
Nigeria's Environment Minister Amina Mohamed said: "For 20 years, Nigeria has not been able to address the vehicle pollution crisis due to the poor fuels we have been importing.
"Today we are taking a huge leap forward - limiting sulphur in fuels from 3,000 parts per million to 50 parts per million."
She said the move would result in major air quality benefits in Nigerian cities and would allow the country to set modern vehicle standards.
The WHO says that pollution is particularly bad in low and middle-income countries.
Five countries in West Africa have decided to stop importing "dirty fuels" from Europe, the UN Environment Programme says.