The breakout came during a riot that officials say may have been prompted by a power cut that knocked out supplies of pumped water.
Hundreds of police and soldiers have been trying to restore order at the prison, and combing the surrounding area for the escaped inmates.
Some of the fugitives were reportedly convicted of terrorism charges.
"The prisoners were annoyed by a blackout and problems getting water, which they said happens often at the prison," Heru Prakoso, a spokesman for North Sumatra province police told Agence France Presse.
"Prison officials told us that the inmates then set fire to the facility."
Some prisoners also reportedly seized guns and hurled bottles at their wardens and, according to police, about 15 prison officials were taken hostage.
The Tanjung Gusta Prison is known to have problems with overcrowding. It is thought to have held 2,400 prisoners despite having an official capacity of 400.
At least 150 prisoners have escaped from a jail in the Indonesian city of Medan on the island of Sumatra.