Michael Mueller asked for Andrej Holm to be fired after his role in the Stasi was revealed by a Berlin newspaper.
Mr Holm, 46, worked for the secret police in the period immediately before the fall of the Berlin Wall in 1989.
But it emerged he claimed in a job application in 2005 that he had never been a full-time employee.
Mr Holm has admitted his full involvement with the Stasi. However, he said he had thought his statement when applying for a post at Berlin's Humboldt University in 2005 was correct.
Mr Mueller said Mr Holm, who is politically independent, had shown he "was unable to look clearly at himself and to draw the consequences".
The Social Democrat mayor added that "especially in Berlin, which was the epitome of a divided city, there must be no doubt about the work to overcome the past" - a reference to confronting the painful legacy of authoritarian rule.
Mr Holm, whose father was a Stasi officer, volunteered to join the agency at the age of 14 and four years later enrolled in its school as an officer cadet before taking a full-time role.
His involvement drew public criticism when he was nominated to be Berlin housing secretary in December 2016 by left-wing party Die Linke, which evolved from the East German communist party.
Mr Holm is not the first German politician to have to stand down because of their ties to the Stasi since the country's reunification in 1990.
Early examples include East German leader Lothar de Maiziere, who was appointed minister for special affairs after reunification, and Wolfgang Schnur, founder of East German political party Democratic Awakening.
Berlin's mayor has dismissed the city's housing secretary over his links to the Stasi, the former East Germany's dreaded secret police.