A judge ruled that the case was not strong enough against the officers who put down the riot in the notorious Carandiru jail.
He said although the evidence was not conclusive excesses had been committed.
No police were injured as they took back the jail but 111 prisoners died.
The trials of most of the officers took place in 2013 and 2014 almost 20 years after the event but none had begun to serve sentences.
The judge, Ivan Sartori, said the prosecution had failed to identify what the police had done when they entered the jail.
"In this case we don't know who killed who and who did what."
The police officers were convicted of having executed prisoners in cold blood although their defence had argued they had only fought back during a violent uprising.
Prosecutors have argued that many of the prisoners were found naked in their cells and had been killed after they had been disarmed and overpowered.
The local prosecutor's office said it was "perplexed" by the ruling and that it would appeal.
The police officers received sentences of between 48 and 624 years in prison but none have served any part of their sentences.
The leader of the operation, Colonel Ubiratan Guimaraes, was sentenced to 632 years in prison in 2001, but was acquitted on appeal in 2006.
He was murdered that same year in suspicious circumstances.
Relatives of more than 100 prisoners who died during a riot in 1992 in a jail in the Brazilian city of Sao Paulo are appealing after a court threw out convictions against 74 police officers.