Images on social media showed the gravestones in Sarre-Union daubed with swastikas and Nazi slogans.
On his Twitter feed, Prime Minister Manuel Valls said every effort would be made to catch the perpetrators of an "ignoble and anti-Semitic act, an insult to memory".
Anti-Semitic attacks have risen sharply in France in recent years.
A kosher supermarket in Paris was one of the sites targeted by Islamist gunmen in a series of attacks last month which left 17 people dead.
A special investigative team is at the cemetery in Sarre-Union, the French news agency AFP reports, quoting Interior Minister Bernard Cazeneuve.
Hundreds of Jewish graves have been desecrated at a cemetery in eastern France, near the border with Germany.