Why do you think the "rock quarry" wasn't sacred?  There are dozens of shrines scattered throughout the quarry.  There is pretty good evidence that the adze makers using the quarry were organized as a guild whose authority was legitimized through ritual appeal to the gods for favor.

Also, there are dozens of shrines on the mountain outside the quarry.  I don't have the exact number at hand, but if I remember correctly, more than 100 shrines have been recorded on the mountain by archaeologists.