Media playback is not supported on this device
The substitute struck powerfully from 25 yards to give Celtic a 1-0 win at Rugby Park and ensured Deila's men would remain top of the Premiership.
"It is the best way to win sometimes," Deila told BBC Scotland.
Killie boss Lee Clark, meanwhile, described the defeat as "the hardest one I have had to take".
"If there was a team that deserved to win the game, it has to be us, without a shadow of a doubt," said the Englishman. "We certainly didn't deserve to lose.
"I am just absolutely devastated to lose the game.
"You can see all aspects of our game have improved, and we were very good without the ball. But a team in our predicament needs points.
"They couldn't have given any more or played any better. I wouldn't have thought there was a better all-round team performance all season.
"It is galling to lose the game. The only crumb of comfort was it was a wonder goal that won it for them - it was unstoppable."
The Ayrshire side remain eight points above bottom side Dundee United, who have two games in hand including their city derby against Dundee on Sunday.
"We will stay up, no problem," insisted Clark. "I have got no doubt about that. We will stay in the league, and we will go on to bigger and better things once we do that."
Deila admitted the champions were not at their best.
"We didn't play well at all but sometimes you just have to carry the points away with you and that is what we did," the Norwegian said. "That is the most important thing.
"We were more passive and didn't get up to tempo with our passing. We were struggling with the surface, which is not an excuse, but it is something we have to work on.
"He [Rogic] has done it a few times in training this week, he has been really good. We were always thinking he could come on and do it. What a feeling."
Celtic now know they will resume league duties against Hearts on 2 April after the international break top of the league.
"The fans are behind Celtic, it is not about me," added Deila. It is about Celtic and you can see the passion. I love the way the fans care about the club."
Celtic manager Ronny Deila said Tom Rogic's last-minute winner against Kilmarnock would be remembered "for a long time".