Mr Faymann came to power in 2008 but has faced criticism within his party since the far right won the first round of presidential elections last month.
He told a hastily convened press conference that Austria needed a chancellor who had his party "fully behind him".
He called for the government to make a new start.
"I'm stepping down from my role as chancellor and SPO leader," he said, acknowledging he had lost support within the party.
In his statement he singled out the challenges of bringing down unemployment, as well as social cohesion and the refugee crisis.
Earlier this year, Mr Faymann yielded to pressure from a conservative coalition partner to cap the number of people allowed to claim asylum in Austria after the arrival in 2015 of more than 90,000 asylum seekers.
That decision prompted criticism from trade unionists and the SPO's youth wing, BBC Vienna correspondent Bethany Bell reports.
But there was further pressure on him after far-right Freedom Party candidate Norbert Hofer won the first round of the presidential election a fortnight ago. The green candidate was second while Mr Faymann's Social Democrats and the centre-right People's Party failed to make it to the run-off vote on 22 May.
"I'm firmly convinced that this country is strong enough to face the challenges of the future," the outgoing chancellor told reporters.
Austrian Chancellor Werner Faymann has resigned after losing the support of Social Democratic party colleagues.