The Tower Gardens Pavilion, which dates back to 1879, is in danger of collapsing, according to Skegness Town Council.
It said much of the building was rotten and beyond repair.
Plans for a new community building would include a heritage centre.
More on this and other stories in Lincolnshire
Previous attempts to save the building, which has been empty since 2007, have been unsuccessful, and a recent survey found rising damp, corrosion, holes in the roof, collapsed floors and rotting woodwork.
Steve Larner from the town council said a new building would help with the regeneration of the area, but acknowledged some would be disappointed to see the pavilion demolished.
"Unfortunately it is not possible to save the existing building, the state of decay is such that the wood is rotten, the floors are rotten, it could all fall down," he said.
Public meetings will take place in the town hall later and at the the site of the pavilion on Thursday. There is also an online survey.
Under the plans, the responsibility and management of the site would be transferred from East Lindsey District Council to Skegness Town Council.
Any new building would have a focus on sustainability and generate its own income, the council added.
Public meetings are set to take place on plans to demolish a "rotten" Victorian pavilion in Skegness and create a £3.5m community hub in its place.