The 31-year-old came to Brazil as the most decorated Olympian in history and has now won 23 medals at five Games.
He swam the second leg for the USA, who won in a time of three minutes 9.92 seconds, with France second in 3:10.53 and Australia third in 3:11.37.
Phelps came out of retirement in 2014, having quit after London 2012.
He has won twice as many Olympic gold medals as the athlete second on the list, former Soviet gymnast Larisa Latynina.
Find out how to get into swimming with our special guide.
"On the block, I thought my heart was going to explode," said the record-breaking American. "I was so hyped, so excited."
Subscribe to the BBC Sport newsletter to get our pick of news, features and video sent to your inbox.
Michael Phelps won a record 19th Olympic gold medal as the United States clinched the men's 4x100m freestyle relay title in Rio.