The petition, organised by political group Momentum Movement, may force a referendum, which Balazs Furjes says would be "too little, too late".
Los Angeles and Paris are the other bidding cities after Hamburg pulled out in 2015 followed by Rome a year later.
"This ship has sailed, it seems," Furjes told Inforadio.
"Unity has gone up in the air, and lacking that, we stand no chance against Paris or Los Angeles," Furjes added.
The International Olympic Committee (IOC) will elect the winner at a vote in Lima, Peru, in September 2017.
"Over a quarter-of-a-million signatures have come together so this money is spent on modern hospitals and well-equipped schools instead of on the Olympics,'' said Andras Fekete-Gyor, chairman of Momentum Movement.
Hungary is the only one of the 10 most successful medal-winning countries at the summer Olympics never to have hosted the event.
Furjes added: "The Hungarian Olympic family, the capital, Parliament, and the government have all made unified decisions.
"Now I see that cause ran aground, even though a 120-year-old Hungarian dream had a palpable chance of coming true."
Budapest stands "no chance" of hosting the 2024 Olympics, according to the bid's chief organiser after more than 260,000 signed a petition against it.