Cosgrove, resuming on 99 not out, reached 122 before he was bowled by fellow Australian Mitchell Claydon.
Dexter made 131 in a 217-run third-wicket stand with Cosgrove to help Leicestershire declare on 401-9, leading by 137 runs.
Daniel Bell-Drummond and Joe Denly fell cheaply, but Sam Northeast and Sean Dickson saw Kent to 85-2 at stumps.
Leicestershire director of elite performance Andrew McDonald told BBC Radio Leicester:
"In the end the weather had the final say and we ran out of time, but I was really impressed with the consistency we showed on the back of the performance against Glamorgan last week.
"The guys stuck at their task, we got ourselves into the driver's seat, and I am very pleased with the style of cricket we're playing at the moment.
"Now we head to Sussex, and it will be a good measuring stick of where we stand."
Kent batsman Sean Dickson told BBC Radio Kent:
"The boys would obviously have preferred a win, but we'll take it on the chin and move forward - there's a lot of positives to take out of the game as well as negatives, and we can work on the negatives.
"I'm not sure I've ever played cricket in conditions as cold as that though. Some of the guys had seven layers on, as well as woolly hats, and were still struggling.
(On his handled the ball dismissal) "As for the first innings, I wasn't really thinking too straight, it just happened so slowly - I thought what I was doing was right, and as I did it I realised it wasn't."
Neil Dexter and Mark Cosgrove hit tons as Leicestershire earned maximum batting points in a draw with Kent.