Zafar Ansari and Aaron Finch advanced their stand to 85 before David Willey' yorker bowled the former for 40.
Finch reached 52 before driving Steven Patterson to cover and Willey then had Jason Roy caught behind for nought.
Steven Davies (29*) and Ben Foakes saw Surrey to 164-5, but the rain meant only 16.4 overs were possible.
The most spectacular moment came shortly after Australia batsman Finch reached his half-century off 53 balls, when Willey's perfect full-pitch delivery sent two of Ansari's three stumps flying out of the ground.
However, with less than 50 overs bowled in the match so far, a draw looks inevitable unless the weather improves and the two captains are able to engineer a final-day run-chase.
Surrey lost three wickets in successive overs before rain brought a premature end to the second day's play against Yorkshire at The Oval.