A priory of Canons Regular was established on the site in 1153, a few miles from the sea in the northern part of Norfolk and it grew in importance over the following centuries. Founded in the time of Edward the Confessor, the Chapel of Our Lady of Walsingham was confirmed to the Augustinian Canons a century later and enclosed within the priory. From the first, the shrine was a famous place of pilgrimage and the faithful came from all parts of England and the Continent until the destruction of the priory under King Henry VIII in 1538. To this day the main road of the pilgrims through Newmarket, Brandon and Fakenham is still called the Palmers' (Pilgrims') Way.