The U's went ahead when Phil Edwards swept in from close range and Marvin Johnson's deflected cross-shot doubled their lead after the break.
Isaac Vassell pulled one back straight away with a shot on the turn and ex-Oxford striker Danny Hylton nodded in to level with eight minutes remaining.
But Johnson's fine strike from the edge of the box won the match for Oxford.
Oxford, who lost 3-2 to Barnsley in the final of last season's competition, will play the Sky Blues at Wembley Stadium on 2 April.
The match at Kenilworth Road was an entertaining encounter, with 35 efforts on goal.
Vassell had hit the crossbar for the hosts in the early stages before Luton fell behind, and Johnson cleared a header from the Hatters forward off the line with the score at 1-0.
Luton looked to have secured at least a penalty shootout after Hylton, who left Oxford for the Hatters in the summer, netted his 20th goal of the season but Johnson's goal from a quickly taken short corner proved the difference.
Even after that, U's keeper Simon Eastwood had to be alert to keep out an angled drive from Jack Marriott in the final minute of stoppage time.
Oxford United will face Coventry City in the EFL Trophy final following a thrilling win at Luton Town.