Eight fire appliances attended the refinery and police officers were sent to the site shortly after 12:00.
Refinery owners Ineos said there were no casualties as a result of the incident.
Ineos said a leak was detected on a pipe carrying ethylene gas in its Kinneil Gas manufacturing plant which was being isolated.
The company said in a statement: "Staff were evacuated from the immediate area and our well-practiced emergency procedures were implemented with the incident management team being mobilised.
"Measures were put in place to contain the leak in the immediate vicinity of the affected plant and as an ongoing precaution road closures remain in place.
"All of the people working in the area have been accounted for and there were no injuries.
"Other areas within the Grangemouth complex, including the adjacent refinery were unaffected by the incident.
"A full investigation into the cause of the incident is under way.
"The regulators have been kept fully informed throughout."
Pupils at primary and secondary schools in Grangemouth were kept indoors over lunchtime as a precaution at the request of the police, a Falkirk Council spokesman said.
A spokesman for Scottish Fire and Rescue Service said: "At 12.12 on Tuesday, 2 May the Scottish Fire and Rescue Service was alerted to reports of a gas leak at the Grangemouth petrochemical site."
Police Scotland said it attended the incident at about 12:15 and that it had closed a number of access roads.
A full investigation has been launched into the cause of a gas leak at the Grangemouth petrochemical site.