A security official told the AP news agency that most of the escapees were facing serious charges.
Another report said a riot had taken place inside the al-Kwafiya prison.
The jailbreak comes a day after the assassination of a prominent political activist triggered protests in the city, although it is not clear whether the two are connected.
Benghazi is one of the most unstable parts of post-revolution Libya. Last year the US ambassador and three other Americans were killed there.
On Saturday protesters attacked offices linked to the Muslim Brotherhood following the assassination of the prominent political activist Abdelsalam al-Mismari. Hundreds of mourners attended his funeral.
A security spokesman in Benghazi told the BBC that some of those who escaped from the prison had been captured but he did not specify how many.
In a news conference, the Libyan Prime Minister Ali Zeidan said residents in the southern area of the city had stormed the prison in protest at its existence in their district.
The inmates include a small number of people thought to be supporters of Col Muammar Gaddafi, the BBC's Rana Jawad reports. The former leader was toppled and killed in an uprising in 2011.
The Gaddafi supporters had been charged with attacking security bases after the conflict, our correspondent says.
AFP news agency quoted a security official saying that there had been unrest inside the prison before the breakout.
"There was a riot inside al-Kwafiya prison, as well as an attack from outside," he said.
"Special forces called in as reinforcements were given orders not to fire at the prisoners."
About 1,200 inmates have escaped from a jail in the restive Libyan city of Benghazi.