The research, byMore in CommonandPower to Change, suggests less than a quarter of people surveyed think their high street is improving.
It says while support for levelling up is strong, "broken" is the word people most often associate with Britain.
But the government says levelling-up funding is transforming communities.
The government says it has committed Â£13bn to levelling up, "supporting projects to improve everyday life for people across the UK - regenerating high streets, local transport and cultural and heritage assets".
The former Prime Minister Boris Johnson pledged to level up the country when launching the Conservative Party manifesto ahead of his landslide general election victory in 2019.
In the years since, the state of local high streets has become strongly symbolic of the levelling-up agenda.
But of the 2,052 people polled by More in Common and Power to Change, just 23% said their high street was improving, while 32% said it was getting worse.
The research has been seen byBBC Radio 4's Westminster Hourprogramme
The report says the success of the levelling up agenda "will be measured, to a large extent, by the ways in which it is able to deliver positive outcomes for high streets across the country".
More in Common's UK director, Luke Tryl, thinks if high streets are not regenerated, that could lead to a sense that the pledge has not been delivered.
"High streets have become, for many people, the symbol of community decline," he says.
"And the problem is that since the Conservatives embraced the levelling-up agenda in 2019, people are still more likely to say that their high street is neglected and getting worse than getting better.
"That poses a real challenge because people are starting to think, was this talk of levelling up and high street improvement really just another politician's promise made to be broken?"
The research found seven out of 10 people had heard of levelling up, far higher than many other government policies.
About two in five Britons (38%) said levelling up should be either the government's top priority or one of its top priorities, according to the survey.
But levelling-up projects aimed at reducing regional inequality have been beset by delays, asa reportby the National Audit Office found this week.
Mr Tryl believes that the success of the policy in cutting through to the public could come with a catch.
He says the "failure to deliver on levelling up would be a sign that actually maybe the government isn't interested in my area or my community, that actually things aren't going to get better.
"And trust in the whole of politics will suffer as a result."
Rachel Wolf, an author of the 2019 Conservative manifesto and a founding partner at polling firm Public First, says levelling up began as a way of enacting the political change that the Conservative Party believed Brexit voters wanted.
"There was a desire to demonstrate to people who'd voted Brexit partly because they felt that their area was rundown, getting a bit worse with every passing year, that things were going to change," she says.
But Ms Wolf believes that definition broadened in the years that followed and in Westminster came to be a "very grand project to solve all the economic ills of the country".
Four years on, Ms Wolf believes the original promise to change politics has yet to be delivered.
She says: "I do feel desperately sad that a huge number of people voted for Brexit in 2016 because they fundamentally wanted change. And it's very hard to say to them that change occurred."
On the high street in Stroud, the gaps left by shuttered national shops are a visible reminder of those economic ills.
They are likened to "tombstones" by Tony Davey, a local business owner and chairman of the Stroud and District Chamber of Trade and Commerce.
He says high inflation and flagging consumer confidence have "created a perfect storm, which has made it one of the most challenging times I've known to trade on our high streets".
Mr Davey says independent businesses like his Party On Up shop are "resilient", with some new openings in Stroud.
But as for levelling up, he says it is not a "silver bullet" and the government could be taking more urgent action - such as reforming business rates - to revive high streets.
The British Chambers of Commerce says changing shopping habits are presenting communities with "real challenges in maintaining thriving high streets".
Its policy manager, Jonny Haseldine, is urging Chancellor Jeremy Hunt to take "bold action" in next week's Autumn Statement "to boost business investment and help local economies".
The report by More in Common says for people to feel that levelling up has been delivered, the government must improve public services, as well as carrying out local regeneration projects.
More in Common's research found that the word people most closely associated with Britain was "broken".
But when asked to describe their local area in a word, the public's responses were much more positive, featuring words such as "good" and "friendly".
Mr Tryl believes that offers a potential solution to politicians.
"Even people who say their area has been neglected still feel proud of that area," he says.
"So use that as the building block for national renewal, rather than always thinking about everything in a very top-down way."
With a general election expected next year, Labour is keen to highlight what it sees as the failures of levelling up, as the party positions itself as a government-in-waiting.
Angela Rayner, Labour's deputy leader and shadow levelling up secretary, says levelling up "has been exposed as a sham and a scam".
She says Labour would devolve power to communities through a law it is calling - in a nod to the Brexit slogan - the Take Back Control Act.
The new law would form "the cornerstone of Labour's mission to rebuild Britain", Ms Rayner says. "It will deliver on our mission to build an economy where growth boosts living standards and good jobs in every part of the country." The government says levelling up is "a long-term programme of reform that sits at the heart of our ambition". "It is breathing life into long overlooked communities, whether it is record investment in town centres or devolving more money and power out of Westminster to the regions," a Department for Levelling Up, Housing and Communities spokesperson said.
You can hear more on this story onThe Westminster Hourat 22:00 GMT on Sunday, and afterwards onBBC Sounds
The UK government's flagship levelling-up agenda is at risk of losing public confidence in its promise to improve high streets, a new report has found.