The medium-pacer claimed 3-32 as the visitors reached stumps on 186-9, 260 behind Middlesex's first-innings' 446.
They added 109 to their overnight 337-4 after losing Sam Robson early for 159.
Adam Lyth (37) and 18-year-old Harry Brook (38) put on 64 in reply but Yorkshire then collapsed despite skipper Gary Ballance's unbeaten 53.
He reached his half-century from 101 balls with a boundary off Ollie Rayner, but the odds are his side will face the possibility of having to follow-on.
Earlier, Ryan Sidebottom (3-69) and Adil Rashid (3-94) were Yorkshire's most successful bowlers as they picked up the last six Middlesex wickets in 36 overs.
The visitors soon lost Alex Lees in reply, and suffered a double blow either side of tea as Higgins had Lyth caught behind and Brook, who was selected after back-to-back hundreds for the second XI, gave a catch to short leg off Rayner.
From 119-3, Yorkshire collapsed to 146-8 in the space of nine overs, with four Yorkshire batsmen bowled, two of them by Higgins.
Middlesex's Ryan Higgins told BBC Radio London:
"To be honest, when the coach (Richard Scott) rang me to say I was going to make my Championship debut, the phone nearly dropped out of my hand.
"We've got a few boys away with England Lions so I'm not sure how long I will stay in the team but I just want to try and contribute.
"I'd have liked to have scored more than 13 because I felt good and, with the ball, it was just a case of trying to back up the big boys. I bowl wicket-to-wicket and the slope here and the pitch, with its lack of bounce, suits me."
Yorkshire batsman Harry Brook told BBC Radio Leeds:
"It's a great experience to make your Championship debut at Lord's. I enjoyed it and got a good start but, having got to 38, I really wanted to kick on, so that was disappointing.
"I have been in good form for Yorkshire seconds so I just wanted to carry on that form.
"There weren't too many demons in the pitch when I batted but towards the end it's started to keep low. We're not in the best position team-wise and if we have to bat again we will need to bat for as long as we can and hopefully get a lead."
Ryan Higgins marked his first-class debut for Middlesex with three wickets to leave Yorkshire in trouble on the second day at Lord's.