The incident - involving a 28-year-old woman - happened near the bowling club in the Moray town's Cooper Park in the early hours of Sunday.
Officers want to trace four men who were in the area at the time.
They also want to speak to a man who was walking a dog nearby at the time, and a two men who were approached by the woman in a distressed state.
Det Sgt Kerry Rigg said: "While the woman was not injured this incident would have been frightening and has naturally left her extremely upset.
"I would urge anyone with information, or anyone that was in the area at the time and witnessed anything unusual, to contact police."
Police want to trace several people who may be able to help after a woman was assaulted in an Elgin park.