The armed gangs are based in a forest, from where they raid nearby villages.
Defence Minister Mansur Dan Ali told the BBC that about 1,000 troops would be deployed immediately, with more would joining later.
Hundreds of people have been killed in villages in and around Zamfara in the last three years.
President Buhari has faced criticism for not focusing earlier on what is being described as the country's third security crisis, behind the Boko Haram insurgency in the north-east and the growing militant attacks on oils pipelines in the southern Niger Delta region.
Africa Live: More on this and other African stories
There are fears that fighters from Islamist militant group Boko Haram may have joined the gangs in Zamfara.
A large swathe of forest spreading from Zamfara to neighbouring states is now referred to as the "New Sambisa", after the forest where Boko Hararm militants are holed up in the north-eastern Borno state, reports BBC Hausa's Aliyo Tanko.
Thousands have been forced to flee their homes following the violence in Zamfara and surrounding states of Katsina, Kaduna, Niger and Kano.
Residents of the affected areas have accused the local authorities of failing to defend them.
Nigeria's President Muhammadu Buhari has sent a special military task force to combat cattle rustlers in the north-western state of Zamfara.