Banners calling for David Cameron to quit were carried by protesters as they marched to Trafalgar Square.
The demonstration, organised by the People's Assembly, was also attended by shadow chancellor John McDonnell.
He said that a Labour government would end cuts and "halt the privatisation of our NHS". The government says austerity measures are key to deficit reduction.
Mr McDonnell also pledged that his party would scrap the work capability assessments affecting people with disabilities and target homelessness by building hundreds of thousands of council homes.
The government was "bankrupt" in its political ideas and handling of the economy, he said, and called for Mr Cameron to resign and to "take his party with him".
"On every front now we are seeing the government in disarray - in terms of the economy we are slipping backwards instead of growing," Mr McDonnell said.
Prime Minister David Cameron has previously argued that by making savings over the course of the parliament the government can "prioritise what matters for working families - schools, the NHS and our national security".
Labour's Diane Abbott, a fellow speaker, said that fighting austerity was the "political struggle of our time" and blamed cuts on "forcing people out of work and into zero hours contracts".
Also present at the rally were Len McCluskey, general secretary of the union Unite, Green Party leader Natalie Bennett and National Union of Teachers general secretary Christine Blower.
At the beginning of the event, which set off from outside University College London, the National Health Singers sang a song which included the lines "don't let our junior docs be worked around the clock", and "help us keep you safe, don't take our rights away".
Chris Nineham, of Stop The War Coalition, said: "Austerity is not about economic necessity, it is a political choice."
The government has pledged to save Â£12bn from welfare by the end of Parliament in 2020 but has abandoned planned cuts to disability benefits after Iain Duncan Smith resigned as work and pensions secretary last month over the proposals.
Thousands of people have taken part in a protest in central London against government cuts.