The firm recently found that managers of a factory in Syria had paid local armed groups in order to stay open.
Mr Olsen said he had not been involved in any wrongdoing, and was standing down to bring "serenity" to the Swiss-French company.
Lafarge is the world's biggest supplier of concrete, cement and aggregates.
In his statement, Mr Olsen said his decision was "driven by my conviction that it will contribute to addressing strong tensions that have recently arisen around the Syria case".
"While I was absolutely not involved in, nor even aware of, any wrongdoing I believe my departure will contribute to bringing back serenity to a company that has been exposed for months on this case," he  added.
In March, the group explained that its own internal investigation had discovered that the company's managers at its Jalabiya plant in northern Syria had, until 2014, paid protection money to various local militias as the political and military situation deteriorated during the country's civil war.
In its latest update, the company confirmed that the payments had indeed been made but agreed that Mr Olsen was not to blame.
"The board has concluded that Eric Olsen was not responsible for, nor thought to be aware of, any wrongdoings that have been identified as part of its review," it said.
The Syrian factory started up in 2010, after an investment of  $680m over three years, and stopped operating in September 2014.
LafargeHolcim was formed in 2015 by the merger of the French Lafarge company and its Swiss rival Holcim and now has its headquarters in Switzerland.
Mr Olsen was promoted to chief executive at the time and was today praised by the company for his "invaluable" contribution in overseeing the merger.
"Eric Olsen is a leader who brought together two companies to create one genuinely global champion," said Beat Hess, the chairman of the group.
"We are very grateful to him for his efforts."
The company is now looking for a new chief executive and Mr Hess will act as interim chief executive for the time being.
The head of cement maker LafargeHolcim, Eric Olsen, is stepping down from the role following an investigation into the company's activities in Syria.