Under Mr Anwar, the so-called People's Alliance coalition had made significant gains and posed the biggest threat to Malaysia's governing coalition since the country's independence.
But in the wake of his arrest the three parties clashed over policies.
Prior to the break up, Prime Minister Najib Razak's own ruling coalition had been struggling to win back votes.
The dispute began over an attempt by the conservative Pan-Malaysian Islamic Party (PAS) to enforce a form of Islamic law called hudud.
The more liberal and ethnic Chinese-dominated Democractic Action Party (DAP) objected to the move and the two parties voted to sever ties with one another.
Mr Anwar's multi-faith and multi-ethnic Parti Keadilan Rakyat (PKR) declared on Wednesday that the alliance "no longer functions formally."
The future of the opposition is unclear as a few parties say they will still work with each other and some analysts said a reconciliation was possible.
Mr Anwar was in February jailed for five years after losing his appeal on a sodomy charge, which he claims was politically motivated.
The next general elections are due in 2018.
Malaysia's opposition alliance has collapsed just months after its leader, Anwar Ibrahim, was jailed for sodomy.