Britain's Murray and Brazilian Soares upset the world number one pairing 7-5 4-6 6-3 at Flushing Meadows.
They will play Spain's unseeded Pablo Carreno Busta and Guillermo Garcia-Lopez in Saturday's final.
Murray and Soares won the Australian Open title in January - their first Grand Slam tournament together.
The Scot, 30, has now reached the final in four of his last six Grand Slam events, and two of four since teaming up with Soares at the start of 2016.
Murray lost to Herbert and Mahut in last year's US Open final when he was playing with Australian John Peers, and asked what he had learned from  that experience, he said: "Don't lose. It's not much fun to lose in finals.
"We've had a great tournament so far. We'll give it our very best to try to win - it would be a great end to the week."
Soares, 34, added: "I'm very proud, they're a very tough team.
"They're the best team this year, defending champions, playing an amazing level, but I thought we did extremely well.
"We served well and I think we played really well under pressure, which at this stage of the tournament is important."
Carreno Busta and Garcia-Lopez beat fellow Spaniards Feliciano Lopez and Marc Lopez 6-3 7-6 (7-4) in the second semi-final.
Jamie Murray and Bruno Soares beat defending champions Pierre-Hugues Herbert and Nicolas Mahut of France to reach the US Open doubles final.