Patterson dismissed both Somerset openers and captain Tom Abell.
Earlier, Somerset spinner Dom Bess (5-80) was helped by Jack Leach (3-53) as four Yorkshire batsmen fell in 30 minutes after lunch.
Visiting skipper Gary Ballance (98 not out) top-scored but ran out of partners as they were all out for 283.
The loss of experienced duo James Hildreth and Marcus Trescothick was a key blow for the hosts, who need another 161 runs for a first County Championship win of the season.
Somerset will resume on Monday with Adam Hose and Steven Davies, who steadied the ship with a 50-run partnership.
Peter Handscomb (70) was one of the batsmen dismissed by 19-year-old Bess, who has four five-wicket hauls from five County Championship matches this season.
England batsman Ballance seemed destined for a fourth century of the campaign but was left stranded two runs short as Karl Carver was run out and Ryan Sidebottom was trapped lbw for a duck before tea.
At that stage the match looked evenly poised, but seamer Patterson's inspired spell means the Tykes go into the final day as slight favourites.
Three Steven Patterson wickets put Yorkshire in a strong position as Somerset reached 101-4 chasing 262 to win their Division One game at Taunton.