The hill and the surrounding area was long inhabited as evidenced by two socketed axes from the Bronze Age (1100–850 BC) found on Rambynas. Historians believe that the hill was the location of Ramigė, a fort of the Skalvians, one of the Baltic tribes. In 1275, the fort was attacked and destroyed by the Teutonic Order. The vogt of Sambia first attacked Ragnit (Neman), then crossed the river, and captured Ramigė. That signified Teutonic conquest of Skalvia. However, an exploratory excavation of about area in 2002 by found no significant cultural layer.