Bakir Izetbegovic, a member of Bosnia's tripartite presidency, said the appeal would be launched before a 10-year deadline expired on 26 February.
Bosnian Serb officials warned the move would trigger a new political crisis.
In 2007, the International Court of Justice (ICJ) cleared Serbia of direct responsibility for genocide.
The ICJ found only one act of genocide during Bosnia's 1992-95 war - the massacre of about 8,000 Bosnian Muslim (Bosniak) men and boys by Bosnian Serb forces in the town of Srebrenica in 1995.
It was Europe's worst atrocity since World War Two.
The ICJ also ruled that Serbia had violated international law by failing to prevent the killings.
On Friday, Mr Izetbegovic said the appeal would be submitted by Bosnia-Herzegovina's legal representative.
"Everyone needs the truth, even those who oppose it, a truth that will be written by international judges, experienced and impartial," he said.
But Bosnian Serb officials said such a move could not be made without consensus within the country's presidency, where a Bosnian Serb and a Bosnian Croat take the other two seats.
Bosnian Serb leader Milorad Dodik also urged ethnic Serb politicians to challenge the legitimacy of any appeal.
In Serbia, Prime Minister Aleksandar Vucic described Mr Izetbegovic's announcement as "bad" for relations between the two countries.
Politicians from Bosnia's two semi-independent entities - the Muslim-Croat Federation and the Republika Srpska - have often clashed, triggering political crises.
The powers of the central government in the capital Sarajevo are very limited.
Bosnia will next week ask the UN's top court to review its ruling which cleared Serbia of genocide in the 1990s, Bosnia's Muslim leader has said.