Two first-half goals from Kemar Roofe steered the League Two side to victory over their League One opponents.
David Forde spilled Roofe's near-post shot through his grasp for the opener.
Millwall striker Steve Morison saw a close-range header saved before FA Cup hero Roofe nodded home after John Lundstram's effort had hit the bar.
Roofe also scored twice at the weekend as the U's knocked Premier League strugglers Swansea out of the FA Cup.
His first against Millwall was his 16th of the season, but probably the most fortuitous as Forde spilled the ball into the net.
Roofe was then quickest to react to Lundstram's curling strike, stooping to head home as the ball rebounded off the underside of bar just before the break.
Oxford claimed a late penalty while Benji Buchel saved well from Morrison and Jed Wallace to maintain their two-goal cushion.
Oxford United will hold a two-goal advantage over Millwall going into the Johnstone's Paint Trophy southern area final second leg at the Kassam Stadium.