Ex-congressman Odin Sanchez has been held by the National Liberation Army (ELN) since he offered to swap places with his ailing brother in April.
President Juan Manuel Santos said Mr Sanchez had to be released before peace talks could start in Ecuador.
The ELN has reacted angrily.
Chief negotiator Israel Ramirez, also known as Pablo Beltran, accused the government of changing its requirements "at the last minute".
"This gentleman will be released during the first round of talks," Mr Beltran told a Colombian radio station. "That is what we committed to do."
On Thursday the Red Cross and Roman Catholic Church began working with the guerrilla group to release Mr Sanchez but it was not clear when the operation would finish.
How significant is Colombia's ELN rebel group?
When Mr Ramirez was interviewed on Blu radio station, he refused to be drawn on how many hostages the ELN still had, saying only that the numbers were "very few".
Mr Sanchez had been thought to be the last. He handed himself over to the rebels in April in exchange for the release of his brother Patrocinio, a former governor of north-western Choco province, held by the rebels for almost three years.
The government is struggling not only to salvage the peace talks with the ELN but also a deal struck with Colombia's largest rebel group, the Revolutionary Armed Forces of Colombia (Farc).
The Farc and the government signed a peace deal last month but the agreement was rejected by Colombians in a popular vote on 2 October.
More than 260,000 people have died in Colombia's armed conflict which has pitted left-wing rebels against right-wing paramilitaries and the security forces.
The second-biggest rebel group in Colombia has clashed with the government over when to release a hostage whose freedom is a key condition for peace talks.