Di Matteo's side bounced back from successive defeats with a 3-0 win over Rotherham but conceded a late equaliser against Huddersfield on Tuesday.
"We're a work in progress. The players that have come in need to understand how we want to play," the Italian said.
"It's going to take time but it's a shame because I thought we deserved a bit more [against Huddersfield]."
Villa were beaten 1-0 by Sheffield Wednesday in their first game back in the second-tier since 1988 and were knocked out of the EFL Cup by League Two side Luton.
Rudy Gestede scored twice in their win against the Millers on Saturday, but Villa were forced to settle for a point against Huddersfield when keeper Pierluigi Gollini's clearance bounced off Michael Hefele's back into an empty net in the 86th minute.
"There is a lot of improvement in the team in the weeks that we have worked together," the Villa manager told BBC WM.
"I think it's games like this you need to be able to score a second goal to kill the game or be able to see it out and get the three points."
Aston Villa boss Roberto di Matteo says they need to time to turn around their fortunes this season.