St Albans council aims to turn its 19th neo-Palladian town hall, called "not fit for purpose", into a cultural hub.
Architect George Smith designed the Grade II* listed building in 1831.
The plan is to turn it into a window on the city's history while staging cultural exhibitions with the help of the Heritage Lottery Fund (HLF).
The council must raise £1.7m to gain access to funding from the HLF.
Annie Brewster, cabinet member for heritage at St Albans City and District Council, said: "Since it was built the building has not been fit for purpose."
She said it had been designed for all but only a minority of residents were catered for.
"Georgian ladies wanted a grand 60ft (18m) ballroom," she said.
"Magistrates wanted a double height central courtroom with cells beneath to vie with Shire Hall in Hertford.
"These features are all still here in a building four million people walk straight past each year and it has not been used properly for about 25 years.
"Now the city has a chance to create a magnificent cultural hub."
Architects John McAslan & Partners plan to create a walkway around the building to encompass views of the medieval city towards the historic Clock Tower and the cathedral.
A changing exhibition programme will reflect the city's heritage since its early Roman settlement.
This includes Alban, the first British saint, and Nicholas Breakspear, the first and only English Pope.
The country's first hot cross bun originated at the Abbey in 1381.
The city is the birthplace of the Ryder Cup golf competition and former residents include scientists Sir Francis Bacon and Prof Stephen Hawking and film director Stanley Kubrick.
A campaign has been launched to raise £7.75m for a new museum to bring hundreds of thousands of visitors to a Hertfordshire city centre.