The troops and their allies, backed by Russian air strikes, have reportedly seized part of a neighbourhood in the west of the city.
The jihadists recaptured the Unesco-listed archaeological site in December from government forces.
Parts of the ancient city have been destroyed by the militants.
There were clashes and heavy shelling across the historic city as the offensive unfolded, the UK-based monitoring group Syrian Observatory for Human Rights said.
A media unit run by the allied Lebanese Hezbollah movement said earlier that the pro-government forces had reached the citadel, which sits on a hill overlooking the famous Roman-era ruins.
The Syrian government troops also seized a strategic area known as the "Palmyra triangle", the official Sana news agency reported.
IS held the ruins and the nearby city, known locally as Tadmur, for 10 months after seizing it for the first time in May 2015. It blew up temples, burial towers and the Arch of Triumph, believing shrines and statues to be idolatrous.
They also destroyed the Temple of Bel - the great sanctuary of the Palmyrene gods - which had been one of the most important religious buildings of the 1st Century AD in the East.
The militants were then forced out by a Russian-backed government offensive in March 2016, but regained control while pro-government forces where focused on battling for the city of Aleppo late last year.
In January, satellite images revealed that the group destroyed the tetrapylon - a group of four pillared structures which were mainly modern replicas - and part of the Roman Theatre.
Meanwhile, the US said positions held by the their allies of the Syrian Arab Coalition were bombed by Russian and Syrian aircraft near the IS-held town of al-Bab, in northern Syria.
"I believe they thought were held by ISIS, yet they were actually - on the ground - were some of our Syrian Arab coalition forces," Lt Gen Stephen Townsend told reporters, using an acronym for IS.

Sum: Syrian government forces have entered the ancient city of Palmyra, pushing back militants from so-called Islamic State (IS), activists say.


Media playback is unsupported on your device
11 November 2014 Last updated at 02:44 GMT
It's China's "Singles Day" -  which started as an excuse for the country's young singles to spoil themselves and do a little online shopping.
But now it has turned into the world's biggest day for online retail.
Leisha Chi reports.
Watch more reports on Asia Business Report's website.

Sum: Online retailers in China are gearing up for what promises to be the most lucrative day of their year.


No deal has been agreed yet but the Italian side want to bolster their squad before they resume after their winter break on 8 January.
Lucas, 29, is not Inter's only option but club sources have indicated there is a "strong possibility" of the move going through.
The Brazilian is Liverpool's longest-serving player.
Lucas joined Liverpool from Brazilian side Gremio in 2007 and has been capped 24 times by his country.
He has played 324 games in all competitions for the Anfield side but has started only three games in the Premier League this season.

Sum: Inter Milan are keen to sign Liverpool midfielder Lucas Leiva on loan for the remainder of the season.


The recently-elected Communist government says the 14.5% tax is aimed at making people more conscious about food choices and curbing obesity.
"This is more of a preventive measure as Kerala's food habits are changing dramatically. People are eating a lot of junk food and rejecting traditional food," says Finance Minister Thomas Isaac.
Kerala has the most number of people suffering from obesity after the northern state of Punjab in India, according to a national family health survey. With increasing affluence, lifestyle diseases are on the rise and the government aims to check this with the fat tax.
But the southern state is a relatively new stop for global fast food chains like McDonald's, Burger King, Domino's and KFC.
McDonald's, the world's biggest fast food chain, has only seven restaurants in Kerala and Burger King just launched its first outlet in Kochi a few weeks ago.
While Burger King didn't want to comment on how this newly imposed tax will affect it, an official statement from McDonald's said the company would be studying the details of the proposal.
Local cafe owners though are more vocal about the proposed tax.
Isaac Alexander, owner of a popular cafe in Kochi, says the "ambit of taxation is very narrow".
"A lot of local food is more fatty and unhealthy. I don't think the tax is a bad thing but it has to be comprehensive and acceptable to all the stakeholders," says Mr Alexander.
"Otherwise it is discriminatory. Just because you serve pizza and burger doesn't mean other people are serving healthy food."
This is a sentiment echoed by Annie Joseph, a self-professed fast food enthusiast.
"We already pay a number of taxes. This [fat tax] will just add to that. Instead the government should promote health food and bring down their prices," she says.
But the government isn't unduly worried about the criticism and says it is targeting only the "elite section of the society" with this tax and that its main aim is to improve public health and get people thinking on what they eat.
Levying taxes on high calorie food items and drinks is one way to rein in consumption.
Denmark introduced a fat tax in 2011 but repealed it by 2013 when it found consumers shopping across the border for high fat goods.
Hungary taxes foods high in sugar, salt and fat. Mexico taxes sugary drinks, breakfast cereals and sweets. In the US, battles are being fought over taxes on sugary drinks. Philadelphia became the first major city in the US to introduce a soda tax.
Meanwhile, the Kerala government has plans to impose the tax on more food items, including refined flour goods and sugary drinks in the near future.
"The fight against fat has just begun," says minister Thomas Isaac.
Supriya Menon is an independent journalist based in Kerala

Sum:
Kerala is the first state in India to introduce a "fat tax" on burgers, pizzas, doughnuts and tacos served in branded restaurants.