The ELN was reacting to a demand by government negotiator Juan Camilo Restrepo for the release by Thursday of a man held hostage by the ELN.
Mr Restrepo said the talks would not begin if the hostage was not released.
The two sides have been engaged in an armed conflict for five decades.
Government and ELN rebel negotiators announced back in March that they would open formal peace negotiations.
How significant is Colombia's ELN rebel group?
The talks, originally scheduled to start in May, were delayed after the ELN rebels failed to meet the government's demand that the guerrillas stop kidnapping people.
After the ELN made a commitment not to carry out any more kidnappings, the two sides set a date of 27 October to start formal talks in neighbouring Ecuador.
But with only two days to go to the meeting in the Ecuadorean capital, Quito, the issue has flared up again.
Mr Restrepo said on Monday that unless the ELN released the man who is believed to be their last remaining hostage, the talks would be off.
"If Odin Sanchez isn't released safe and sound between now and Thursday, the conditions will not be in place to begin the public phase of the negotiations," Mr Restrepo told Colombian radio.
Odin Sanchez handed himself in to the rebels in April in exchange for the release of his brother Patrocinio.
The former governor of north-western Choco province, Patrocinio Sanchez, was held by the rebels for almost three years and had fallen ill when his brother suggested the swap.
Patrocinio Sanchez said on Monday that he opposed the ultimatum given by the Colombian government as he said he feared it could put his brother in further danger.
The government is struggling not only to salvage the peace talks with the ELN but also a deal struck with Colombia's largest rebel group, the Revolutionary Armed Forces of Colombia (Farc).
The Farc and the government signed a peace deal last month but the agreement was rejected by Colombians in a popular vote on 2 October.
Farc and government negotiators are now trying to reach a new agreement which is acceptable to those who voted "no".
More than 260,000 people have died in Colombia's armed conflict which has pitted left-wing rebels against right-wing paramilitaries and the security forces.
Colombia's second largest rebel group, the National Liberation Army (ELN), has accused the Colombian government of "torpedoing" peace talks between the two sides due to start on Thursday.