Somerset were all out for 438 as James Hildreth fell lbw to Keith Barker (4-94) for 118 and Luke Ronchi made 51.
But the visitors collapsed from 105-1 to 105-6 in the space of five overs, with 24-year-old spinner Leach claiming four of those wickets as Somerset's bowling bonus points guaranteed safety.
Warwickshire closed on 180-7.
After Hildreth failed to add to his overnight score, Leach shared a 61-run last-wicket stand with Jamie Overton (40) to ensure Somerset picked up maximum batting points, finishing unbeaten on 21.
Warwickshire captain Varun Chopra then hit an assured half-century and shared an 88-run first-wicket partnership with Ian Westwood before the latter was bowled by Leach (34).
When England batsman Ian Bell was trapped lbw by Alfonso Thomas for 12, it triggered the collapse as Leach had Chopra caught for 54 before removing Freddie Coleman, Tim Ambrose and Rikki Clarke for ducks in the space of 13 balls.
Somerset spinner Jack Leach:
"My main thoughts were on winning the match, so it was a while before I realised that taking my fifth wicket meant we were safe from relegation.
"It was quite surreal when all those wickets fell without a run being scored. The pitch was helping me a bit, but not turning a great deal.
"I also enjoyed contributing some runs to the last-wicket stand with Jamie Overton because reaching 400 was one of our main aims going into the match."
Warwickshire director of cricket Dougie Brown:
"It was the second time this season we have lost five wickets for no runs. We did the same against Yorkshire at Edgbaston, so it was pretty disappointing for it to happen again.
"We bowled and fielded really well at the start of the day before that last-wicket partnership. Then to get to 88 for none and 105-1, to find ourselves 105-6 was very frustrating.
"It's very dry and the ball is spinning from straight. But we have a partnership between Jeetan Patel and Laurie Evans to build on and we have to learn from the way they have played."
Jack Leach took 5-47 as Somerset ensured their Division One status for next summer by dominating day two against Warwickshire at Taunton.