The former London mayor, 70, has been accused by Labour's Alan Quinn of giving "grotesque offence" in Prestwich, with his Hitler comments.
The area has the UK's second biggest Jewish population and Labour lost seats to the Conservatives and Lib Dems.
Mr Livingstone said it was others in Labour who made an issue of it.
He said party MPs who had accused him of anti-semitism has "cost us seats all over the country."
Labour only held onto the third seat in Prestwich by 19 votes.
Mr Quinn called for suspended Livingstone to be expelled from the party.
"He's caused grotesque offence and the suspension needs to turn into expulsion," he said.
"This party should not be a place for bigots and racists.
"I was at a Holocaust memorial event a few days ago and I saw the raw emotion, anger and hurt that his crass comments have caused.
"Prestwich has a big Jewish population and a growing Muslim population. It is an example of how a community can live together. Livingstone's comments have damaged that. We need him out of the party."
Mr Quinn was angered after Paddy Heneghan, Labour's cabinet member for children, family and culture, lost the Holyrood ward to Lib Dem Steve Wright.
Conservative David Silbiger took Sedgley from Labour's Andrea Simpson - after a recount - by just 60 votes.
The Liberal Democrats lost control of Stockport after their leader was defeated and another defected to Labour an hour before voting ended.
Labour is the largest single party with 23 councillors while the Lib Dems have 21, the Tories 14 and there are five independents.
Stockport's Lib Dem leader Sue Derbyshire lost her seat when she was beaten by Labour's Charles Stewart.
Labour retained Bolton council but lost two seats to UKIP. Elsewhere in Greater Manchester the party retained control of Salford,  Oldham, Tameside, Rochdale, Wigan while the Tories held onto their flagship council in Trafford.
A Labour councillor has blamed Ken Livingstone for the loss of two seats in Bury saying Jewish voters were offended by his comments.