It was the last major town under government control in Idlib province. Correspondents say it may give rebels a route to the ruling elite's heartland.
The Islamist forces, including the al-Qaeda-affiliated al-Nusra Front, began the assault on Thursday.
State media said the army had redeployed outside the town "to avoid civilian casualties".
They quoted a military official as saying that, before withdrawing, troops had engaged in fierce fighting with "armed terrorist groups" who arrived in large numbers from the Turkish border.
Activists from the London-based Syrian Observatory for Human Rights said government forces had fled.
The bodies of at least 60 pro-government fighters were lying in the streets, the observatory added.
Before the rise of Islamic State, the Nusra Front was seen as the strongest and most militant rebel force.
The city of Idlib was overrun by the rebel groups last month.
Since then, government forces have been trying to protect their supply route between Aleppo and Hama province on one side and Latakia on the other.
Latakia is home to the core of the Alawite minority to which the family of Syrian President Bashar al-Assad belongs and is so far largely untouched by the war.
BBC World Service Middle East analyst Sebastian Usher says that if the rebels can drive the government completely out of Idlib province, they would open the way to Latakia.
The town of Jisr al-Shughur has been under government control since the early stages of the conflict. In June 2011, large protests against the regime led to the deaths of 120 troops in circumstances disputed by the government and its opponents.
Islamist rebels in Syria have taken over the key north-western town of Jisr al-Shughur, activists say.