An Alex Cunningham shot deflected into the path of striker Blair Henderson and he scored with a neat half-volley.
There were chances at both ends before Henderson rounded keeper Allan Fleming and slotting home from a tight angle.
Lavery then played the ball wide to Graeme Beveridge and continued his run to head in the full-back's cross.
Campbell, the former Forfar Athletic boss, had been appointed by Arbroath as Todd Lumsden's successor hours before kick-off.
Dick Campbell's reign as Arbroath manager got off to a losing start as Berwick Rangers moved six points clear of bottom side East Stirlingshire.