The 28-year-old was offered a new deal at Argyle but will now join Rovers on 1 July once his current contract expires.
The former Sunderland trainee, who has previously spent four years with Hartlepool, made 80 league starts for Plymouth after arriving in June 2014.
Hartley becomes Darrell Clarke's first new signing of the summer since winning promotion to League One on the final day of last season.
On Wednesday, Rovers' central defender Tom Parkes signed for League Two Leyton Orient on a two-year deal.
Find all the latest football transfers on our dedicated page.
Bristol Rovers have signed Plymouth Argyle central defender Peter Hartley.