Neath Port Talbot council will build the replacement Afan Lido on the neighbouring Hollywood Park site.
Campaigners wanted the new building on the old site but did win their battle to have an eight-lane pool.
It is hoped construction which will take 15 months to complete will start early next year.
"This project is now gaining a tremendous amount of momentum in coming to fruition," Neath Port Talbot council leader Ali Thomas said.
"This will be a wonderful, iconic 21st Century facility we can all be proud of and will be a focal point in enhancing our strategy and vision for regenerating the whole area."
The design features a "wave roof" to reflect its seafront location.
It will have an eight-lane pool with boom and movable floor, a learner pool, 100-seat spectator gallery and four-court sports hall with retractable seating for 100 spectators.
There will also be a fitness suite, dance studio, children's soft play area, a youth club, community meeting rooms and a cafe.
The original Afan Lido was opened in 1965 by the Queen. Graham Jenkins, the brother of the late Hollywood star Richard Burton, was its first manager.
The venue hosted major concerts from Spencer Davies to Pink Floyd in its early days to Coldplay and McFly in recent years.
In December 2009 the building was gutted by fire and finally demolished in late 2011.
A total of 2,628 people took part in a Neath Port Talbot council consultation on replacing the Afan Lido.
Although many people said they would like to see an identical rebuild, the council said just keeping the old lido going was costing £700,000 a year.
Plans have been submitted for a £13.4m sports complex in Port Talbot to replace a lido destroyed by fire four years ago.