Former Gillingham boss Edinburgh replaced Rob Page at a club currently five points above the relegation zone.
In 2016, the Cobblers won League Two and had a 31-match unbeaten run.
"We need an enthusiasm and a smile on people's faces. When we started out in football we did it because of our enthusiasm," said the 47-year-old.
"That's what we've got to have. That smile can become greater and enthusiasm can become bigger when we win games.
"And confidence is key, it's a big part of football. We've got to instil that, get that belief back into the players and I believe I can do that."
Edinburgh watched from the stands as the Cobblers were beaten 2-1 at home by second-placed Scunthorpe United on Saturday, as they continued a run of just two league victories since the start of November.
Former Tottenham defender Edinburgh, dismissed by Gillingham on 3 January, said he was grateful for the opportunity given to him by chairman Kelvin Thomas.
"Kelvin rang me on Sunday and asked me whether I had been scared off (by the defeat) and I said 'no, certainly not'," Edinburgh told BBC Radio Northampton.
"I'm delighted to get this fantastic opportunity. We know what kind of industry we're in, you can sit and twiddle your thumbs for a very long time and not get a phone call, opportunity or interview.
"I've had time to reflect. I was lucky enough to have a holiday away in the short space of time between jobs but I can't wait to get started. Getting out on the training ground on Monday morning was a joy."
Following the arrival of David Kerslake as assistant manager, Paul Wilkinson, Paddy Kenny and John Harbin have all left the Sixfields coaching staff.
New Northampton Town manager Justin Edinburgh says he must put belief back into a side that has lost five of their last six League One matches.