An old chicken farm by the A3 at Thursley next to the Devil's Punch Bowl was used as a construction site for the Hindhead Tunnel which opened in 2011.
Highways England has now said it is a "useful location" for a salt storage depot.
Neighbours fear "horrendous" noise pollution and said officials promised to return the site to agricultural use.
Contractor Kier Highways has applied to build a "winter maintenance" facility to be used by 38-tonne delivery lorries up to 10 times a day.
Tony Kelly, who lives 50m away, said cyclists, horse-riders and walkers use a public bridleway outside the gates.
He told BBC Surrey: "It's unsafe, somebody is going to get hurt, walkers, young kids. It's just the wrong place to have this kind of access.
"Is it really worth impacting an area of outstanding natural beauty and causing distress to the villagers of Thursley when (Highways England) has a compound less than 10 miles away?"
In a statement, Highways England said: "We used Hindhead Hill Farm for our site compound while we were building the Hindhead Tunnel and have honoured our commitment to return it to agricultural use after the project was completed in 2011.
"We need a winter maintenance depot to help keep the A3 operating safely during severe weather. We think this farmyard could be a useful location for a barn, which is why we are taking forward this planning application."
Highways officials have been accused of "breaking a promise" over land at a beauty spot in Surrey.