Mr Mugabe, who is 90, drew applause when he denounced colonialism.
He also spoke of the "scourge of terrorism" from Boko Haram and said there needed to be "lasting solutions" to the issue in Nigeria and Cameroon.
Earlier, UN Secretary-General Ban Ki-moon warned of the dangers of leaders clinging to power.
Mr Mugabe has led his country since independence in 1980.
He is subject to travel bans imposed by the US and the EU because of political violence and intimidation in Zimbabwe.
Within Africa he is a divisive figure, with some seeing him as a nationalist hero and others as a despot responsible for gross human rights abuses.
Western diplomatic sources said his election to the mostly ceremonial post of AU chairman was unlikely to have much impact on relations.
In his address to leaders, Mr Mugabe spoke of the need to take advantage of Africa's mineral wealth and agricultural potential, and guard against exploitation by foreigners.
He said African countries wanted relationships with "friends", but "colonialists and imperialists" had no place in the continent.
Later, he received applause when he referred to the long-running territorial dispute in Western Sahara between Morocco and the Algerian-backed Polisario Front.
"As long as our brothers in Western Sahara are under Moroccan occupation we are not totally free," he said.
African leaders meeting in Addis Ababa have chosen the continent's oldest head of state, Zimbabwe's Robert Mugabe, as AU chairman for the coming year.