The Money Advice Trust, which runs the National Debtline service, said that less than a third of people it surveyed had a budget for the festive period.
It added that a third would be borrowing to pay for Christmas costs.
Various advice services say that building up a savings buffer is key to paying for expensive times of year.
Selling unwanted items can also raise some last-minute cash, they add.
"Money worries can have a huge impact on your life at any time - but the fact that they are putting Christmas at risk for up to five million people shows what an extremely difficult time of year this can be," said Joanna Elson, chief executive of the Money Advice Trust.
"This is also, of course, a busy time of year - and it is easy to see why many people don't want to deal with financial problems in December.
"However, our research shows there are millions of people worrying about Christmas finances who could benefit from seeking advice now, to start to resolve their financial problems. Three-quarters of callers to National Debtline tell us they feel less stressed as a result - and often that first step is the hardest to take."
Recent research suggested that Christmas dinners and trees had been falling in price in real terms compared with previous years, but the Bank of England warned that "vigilance" was needed over levels of personal debt which have been accelerating recently.
Matt Barlow, chief executive of Christians Against Poverty, said: "The average annual household income of our clients is around Â£14,000 so it takes very little to push people into a crisis situation and, sometimes, the pressure to spend at Christmas does just that."
Debt concerns at Christmas can be alleviated by seeking advice well before the bills come in, a charity has suggested.