He was one of 16 people arrested across the central state of Hesse, as 1,100 police searched flats, businesses and mosques.
The man is also wanted by Tunisian authorities over a jihadist attack on the Bardo Museum in Tunis.
Twenty-one tourists and a Tunisian were killed in March 2015.
Many of the victims were British, Japanese, French, Italian or Colombian.
Authorities in Frankfurt said Wednesday's operation was the result of a four-month investigation. They had evidence Islamists were preparing an attack, but described the planning as in its early stages.
Those arrested are aged between 16 and 46 and are suspected of helping a foreign terrorist group and of planning serious violence.
Three men were held in separate raids in Berlin late on Tuesday which included a mosque that Tunisian jihadist Anis Amri visited shortly before driving a heavy goods lorry into a crowded Christmas market in December, killing 12 people.
One of the three detained was said to be in charge of the Fussilet mosque, which has been described as a base for radical Muslims in Berlin.
Amri had spent four years in jail in Italy before travelling to Germany but repeated attempts to send him back to Tunisia had failed because officials there said he had no documents.
Police said the Tunisian man picked up on Wednesday was detained at his home in Frankfurt. He had spent 10 years in Germany until 2013 and had returned as an asylum seeker in August 2015.
Since his return he is suspected of setting up an IS support network.
Tunisian authorities have also linked him to a deadly March 2016 attack on the town of Ben Guerdane, close to the Libyan border.
He was in detention in Frankfurt facing extradition for several weeks but was released last November because Tunisian authorities had failed to submit documents in time.
A 36-year-old Tunisian man suspected of recruiting for jihadist group Islamic State has been detained in multiple raids in Germany.