A spokesman said militants had gained ground inside Afghanistan and had to be stopped from extending their influence.
He said the "Khyber 4" operation, backed by the air force, would focus on the mountainous Rajgal Valley area in Khyber Agency.
Pakistan has previously denied the presence of IS on its territory.
However, the growth of the group - often referred to as Daesh - in neighbouring Afghanistan has alarmed Islamabad and the group has claimed several attacks in Pakistan over the past two years.
"This operation was necessary because Daesh is getting established there and we have to stop the influence spreading into Pakistani territory through the Rajgal Valley," military spokesman Lt Gen Asif Ghafoor said.
IS in the region is mostly made up of former members of the Afghan and Pakistani Taliban and Gen Ghafoor reiterated the government's stance that the group has no established presence inside Pakistan.
"There is no organised infrastructure and we shall not allow them to establish themselves," he said.
"As regards inside Afghanistan, yes, Daesh is getting stronger there, is gaining strength. But still, we feel that it is not purely the Daesh which is there in the Middle East."
He said that the operation would first secure the border near Rajgal Valley and forces would then clear the rest of the area. He said multiple "terrorist" organisations with links to recent attacks in Pakistan had safe havens on the other side of border.
The Rajgal Valley and Khyber Agency are part of the volatile Federally Administered Tribal Areas.
IS controls some territory in Afghanistan and has been seeking to strengthen its presence in Pakistan since 2015 when it carried out its first attack in the country.
At about that time, IS announced the establishment of its Khorasan branch - a historical name for Afghanistan and the surrounding areas. It marked the first time that IS had officially spread outside the Arab world.
Pakistan's military says it has launched a major operation against so-called Islamic State (IS) in the north-western region along the Afghan border.