A converted try from prop Brian Mujati gave Sale the perfect start but Pau then led 10-7 as Daniel Ramsay's score followed a Thibault Daubagna penalty.
Sharks centre Mark Jennings crossed to ensure the hosts led at half-time and Joe Ford's penalty made it 17-13.
Pau's Quentin Lespiaucq-Brettes and Sale's Sam James swapped tries before Jonathan Mills secured a bonus point.
There were never more than seven points between the two sides in an extremely close contest, until replacement lock Mills touched down in the final seconds.
The Sharks lost their opening match 30-12 to Welsh club Newport Gwent Dragons, while Pau's first pool match was cancelled following the Paris attacks on 13 November.
Sale: Haley; Brady, James, Jennings, Addison; Ford, Cusiter; Harrison, Taylor, Mujati, Evans, Ostrikov, Lund (capt), Ioane, Fihaki.
Replacements: Neild, Flynn, Parker, Mills, Beaumont, Mitchell, Jeffers, Edwards.
Pau: Malie; Bobo, Vatubua, Votu, Niko; Fernandez, Daubagna; King, Lespiaucq-Brettes, Natsarahvili, Dry, Ramsay, Bernad, Bouilhou (capt), Coughlan.
Replacements: Campo, Hurou, Moise, Domolailai, Butler, Marques, Ratuvou, Dupouy.
Referee: Ben Whitehouse
For the latest rugby union news follow @bbcrugbyunion on Twitter.
Sale Sharks narrowly beat French side Pau to win their first match in European Challenge Cup Pool 2.