Crown Street Library in Darlington will be closed with its services moving to the Dolphin Leisure Centre in the town.
Campaigners say the library was well used by its community and previously staged a protest to save it.
The town council rubber stamped the closure at a meeting on Thursday and said the move will save the Labour-run authority more than £300,000 a year.
Darlington councillor Nick Wallis said: "The council has to save almost £12m a year between now and 2019/20.
"Relocating the library from Crown Street to the Dolphin Centre will save in excess £310,000 a year."
Gemma Macdonald, who campaigned to save Crown Street Library, said: "On active library users we fair really well. Yes, library use has been declining but we need to look at libraries differently."
A spokeswoman for Friends of Darlington Libraries group said the authority was "ignoring" public opinion and pledged to continue fighting.
The council also confirmed that Cockerton Library will become a community run library and the mobile library service will also be discontinued.
A Grade II-listed library is set to close in order to make hundreds of thousands of pounds worth of savings.