Fly infestation from the centre on Slack Lane, Derby, is thought to be caused by 5,000 tones of rubbish awaiting clearance.
Shows Waste Management said the waste will be moved by 7 July - the deadline given by the agency.
The government body has begun a criminal investigation.
The fly infestation, which has been ongoing for weeks, has left more than 100 people in the area unable to cook food in their own homes.
Resident Jenny Warren said the flies are "all over the food and you can't open windows".
Campaigner Diane Hanrahan, who lives in the area and held a public meeting on Monday, hopes to raise money to take legal action against the agency and the company.
Councillor Paul Pegg, of Derby City Council, said: "[Ms Hanrahan] is trying to seek legal action, to get representation for the loss of earnings and what they have had to pay out to get the flies removed from the properties."
The company said it is in the current situation due to another firm which provided it with the waste.
Sam Turton, from Shows Waste Management, said since the problem arose lorries have been removing waste from the site to landfill and insecticides have been applied to control the problem.
It has been estimated that it could take more than 200 lorry trips before the site is completely cleared.
Mr Pegg added that he was "disgusted" with the Environment Agency for not attending the latest meeting.
"They are a disgrace," he said. "They have failed to monitor this site, they need to be held accountable."
The agency told the BBC that it has met with everyone involved at previous meetings and has a plan of action in place.
Residents plagued by flies from a recycling centre said they hope to take legal action against the Environment Agency and the site operator.