The local authority's cabinet made the decision on Tuesday evening in a bid to make an annual saving of Â£286,200.
It voted to approve a report that recommended closing five of the city's 11 libraries if volunteers and other organisations do not take them over.
Cabinet member for culture Satvir Kaur said the move had been forced on the council by "savage cuts" to its budget.
But she said: "I am confident no libraries in Southampton will close, and I urge community groups to work with us to ensure that."
The proposals have faced opposition with an estimated 150 people staging a silent protest through Southampton in March.
The report recommends Bitterne Library, Central Library, Portswood Library, Woolston Library, Shirley Library and Lordshill Library remain open.
But it says Burgess Road, Cobbett Road, Millbrook, Thornhill and Weston libraries should no longer be funded and managed by the council.
Southampton's mobile library service is also under threat.
Ms Kaur said a consultation showed two thirds of Southampton residents supported the council's plans.
Earlier this year BBC Springwatch presenter Chris Packham said making cuts to the city's library service would be a "damaging mistake".
Five Southampton libraries are at risk of closing after councillors voted to stop running them from next year.