The leaders of Egypt, Ethiopia and Sudan signed the agreement in Sudan's capital, Khartoum.
Egypt has opposed the Great Ethiopian Renaissance Dam, saying it would worsen its water shortages.
Ethiopia says the dam will give it a fairer share of Nile waters.
In 2013, Ethiopia's parliament ratified a controversial treaty to replace colonial-era agreements that gave Egypt and Sudan the biggest share of the Nile's water.
Egypt's then-President Mohamed Morsi said he did not want war but he would not allow Egypt's water supply to be endangered by the dam.
Mr Morsi's successor, Abdul Fattah al-Sisi signed the deal with Ethiopia's Prime Minister Halemariam Desalegn and Sudan's President Omar al-Bashir.
The three leaders welcomed the "declaration of principles" agreement in speeches in Khartoum's Republican Palace, and watched a short film about the Grand Renaissance Dam that highlighted how it could benefit the region, the Associated Press news agency reports.
Mr Halemariam said he wanted to give an assurance that the dam would "not cause any harm to downstream countries", Reuters news agency reports.
Mr Sisi said the project remained a source of concern to Egypt.
"The Renaissance Dam project represents a source of development for the millions of Ethiopia's citizens through producing green and sustainable energy, but for their brothers living on the banks of that very Nile in Egypt, and who approximately equal them in numbers, it represents a source of concern and worry," he said.
"This is because the Nile is their only source of water, in fact their source of life."
Ethiopia wants to replace a 1929 treaty written by Britain that awarded Egypt veto power over any project involving the Nile by upstream countries.
Ethiopia says the $4.7bn (Â£3.1bn) dam will eventually provide 6,000 megawatts of power.
Egypt was apparently caught by surprise when Ethiopia started diverting the Blue Nile - a tributary of the Nile - in 2013.
Ethiopia says the river will be slightly diverted but will then be able to follow its natural course.
Egyptian politicians were inadvertently heard on live TV in 2013, proposing military action over the dam.
Ethiopia has received strong backing from five other Nile-basin countries - Rwanda, Tanzania, Uganda, Kenya and Burundi.
Three African leaders have signed an initial deal to end a long-running dispute over the sharing of Nile waters and the building of Africa's biggest hydroelectric dam, in Ethiopia.