Rome's Mayor Virginia Raggi, from the populist Five Star party, had said the city had to prioritise matters such as rubbish collection and corruption.
It means only Paris, Los Angeles and Budapest are left in the running after Boston and Hamburg also abandoned bids.
The International Olympic Committee is due to make a decision next September.
Committee chief Giovanni Malago said the decision meant Italy had been "made to look like fools" and would miss out on investment.
Rome's protest mayor Raggi finds it tough at the top
When in Rome shake up the politics
However, the Five Star party had repeatedly said it opposed bidding for the Games before winning control of Rome in June.
The Italian government had supported the bid and the broader Rome municipality, which includes many other small town councils, had indicated it would support the bid to overcome Five Star's opposition.
But Mr Malago said he would not go ahead without the main city council's backing.
Rome also applied to host the 2020 Olympics but pulled out of that race in 2012 because then prime minister Mario Monti feared Italy could not afford it.
Italy's Olympic Committee has officially withdrawn its bid to stage the 2024 Games in Rome after the city council voted to oppose the candidacy.