Harry Schwarz's political career started with his election to the Johannesburg City Council in 1951 for Booysens, which had been said to be an unwinnable seat against the National Party. Schwarz won the seat by 954 votes. Despite being the youngest person in the city council, he became chairman of the council's management committee - the most influential committee on the council. While in the council, Schwarz focused on challenging forced evictions of black and coloured people in Johannesburg, and attempted to improve housing and education. Booysens had once been occupied by Labour Party politician Jimmy Green, who was his wife's uncle, who was first elected in 1920 to the City Council.