DawaFFM, Islamische Audios and An-Nussrah all adhere to ultra-conservative Salafist doctrine.
Police mounted raids against the groups on Wednesday, but no arrests were made.
German officials have said videos produced by DawaFFM partly inspired the man who shot dead two US airmen at Frankfurt airport in 2011.
An-Nussrah was judged to be part of the Millatu Ibrahim organisation, which had been banned in June last year.
"Salafism, as represented by the groups banned today, is incompatible with our peaceful, democratic system," Interior Minister Hans-Peter Friedrich said in a statement.
"DawaFFM, Islamische Audios and An-Nussrah aim to change our society in an aggressive way so that democracy is replaced by a Salafist system and the state of law replaced by sharia," he went on.
Searches were carried out early on Wednesday morning at the addresses of 20 people in the states of Hesse and North-Rhine Westphalia, the interior ministry said.
Computer equipment and propaganda material in German and Arabic were seized, as well as more than 10,000 euros (Â£8,600; $13,000) in cash.
German officials also accuse the groups of encouraging violence against those who do not agree with their views.
"These bans will also serve to protect the vast number of peaceful Muslims. They must not be troubled by conflicts deliberately fomented by extremists," Mr Friedrich said.
There are believed to be about 4,000 active Salafists in Germany, out of a broader Muslim population of well over 4m.
Also on Wednesday, German media reported that police in North Rhine-Westphalia had prevented an attempt by radical Islamists to kill the leader of the far-right Pro-NRW party, Markus Beisicht.
Four people have been arrested, at least one of whom was armed, according to local media.
Authorities in Germany have banned three Islamist groups, accusing them of being "anti-democratic" and seeking to impose sharia law.