About 24 of the rocks, known as Hertfordshire Puddingstone, were removed from the former St Albans Museum grounds.
It is thought they were taken from the garden between the 3 and 14 April.
Hertfordshire Police said the theft of the rare rock was being treated as a heritage crime.
PC Sean Lannon said Hertfordshire Puddingstone was "one of the world's rarest rocks" and part of the county's heritage.
"We are doing all we can to ensure that these stones are returned to the museum," he said.
The force has appealed for witnesses or anyone who may have been offered the rocks for sale to come forward.
Hertfordshire Puddingstone is a naturally occurring conglomerate consisting of rounded flint pebbles bound in silica cement, found mostly within the county.
It is thought to have originated from deposits laid down millions of years ago and is called puddingstone because the flints resemble the plums in a Christmas pudding.
Most of the rocks taken came from the site of the Seventh Day Adventist Church during the late 1970s.
St Albans Museum in Hatfield Road closed earlier this year ahead of its move to a new site in the Town Hall which is due to open next year.
A collection of prehistoric stones thought be about 54 million years old has been stolen from a former museum site, police have said.