The centre, 26, replaces Owen Farrell, who moves to the bench.
Northampton Saints pair Tom Wood and Dylan Hartley take over from James Haskell and Rob Webber at flanker and hooker respectively.
Second-row forward Courtney Lawes, another Saints player, has recovered from a knee injury sustained in last weekend's 28-9 win over Samoa.
Media playback is not supported on this device
Farrell, 23, has been overlooked for the starting XV for the first time since June 2012 after under-performing against New Zealand, South Africa and Samoa.
The Saracens player was picked at inside centre for the game against Samoa after starting in his usual fly-half role against the All Blacks and Springboks.
Bath's George Ford remains at stand-off and will win his sixth cap after impressing against Samoa last Saturday in what was his first Test start.
England head coach Stuart Lancaster said: "It's good to be able to give Billy his first start of the series.
"He took on board the feedback we gave him when he first came into camp and he has worked hard both with us and at Gloucester and deserves his chance.
"Owen is a key part of our squad and I am sure he will make a positive contribution should he get the chance."
Lancaster also praised the contributions of Haskell and Webber in the victory over Samoa.
He added: "Tom (Wood) and Dylan (Hartley) were disappointed not to be in the XV last week and they are desperate to go out and perform on Saturday so that we finish the series on a high."
England: Mike Brown (Harlequins); Anthony Watson (Bath Rugby), Brad Barritt (Saracens), Billy Twelvetrees (Gloucester Rugby) Jonny May (Gloucester Rugby); George Ford (Bath Rugby), Ben Youngs (Leicester Tigers); Joe Marler (Harlequins), Dylan Hartley (Northampton Saints), David Wilson (Bath Rugby), Dave Attwood (Bath Rugby), Courtney Lawes (Northampton Saints), Tom Wood (Northampton Saints), Chris Robshaw (Harlequins, capt), Ben Morgan (Gloucester Rugby).
Replacements: Rob Webber (Bath Rugby), Matt Mullan (Wasps), Kieran Brookes (Newcastle Falcons), George Kruis (Saracens), James Haskell (Wasps), Richard Wigglesworth (Saracens), Owen Farrell (Saracens), Marland Yarde (Harlequins).
Gloucester's Billy Twelvetrees will start for England in Saturday's Test against Australia at Twickenham.