Police closed the M4 eastbound carriageway at Llandarcy for five hours following the incident at about 05:00 BST on Saturday.
The man, driving a white Vauxhall Corsa which hit the nearside barrier, was declared dead at the scene.
Two passengers were taken to Swansea's Morriston Hospital and treated for minor injuries.
An 18-year-old driver has died after his car hit a motorway slip road barrier near Neath.