Ignazio Marino claimed to have been backstabbed in a plot orchestrated by Prime Minister Matteo Renzi.
Mr Marino, 60, had tendered his resignation on 12 October amid an expenses scandal and under pressure from Mr Renzi.
But on Thursday, he withdrew his resignation despite losing backing of their centre-left Democratic Party.
The decision prompted Friday's mass resignation of city councillors.
The former surgeon was elected in 2013 and has been embroiled in a scandal over private dinners allegedly paid for with city funds.
He has denied funds were used inappropriately.
Many Romans complain that municipal services, including rubbish collection and road maintenance, have worsened under Mr Marino's watch.
Mr Marino was also criticised for not stopping an alleged mafia boss being given a glitzy funeral in the city.
But his supporters say he inherited a host of problems from his right-wing predecessor, who is being investigated for corruption.
Italy's interior ministry is expected to appoint someone to run Rome temporarily until fresh elections can be held, possibly next year.
The scandal-hit mayor of Rome has been ousted from office after more than half the city's councillors stepped down.