It was found in Farmhill, in the Foyle Springs area of the city, on Friday morning.
Police have cordoned off the area while the device is examined by Army bomb experts.
"The area remains closed while the object is examined," said PSNI Insp Ray Wilson.
SDLP MLA Mark H Durkan said the alert was causing "absolute havoc for local people, many of them elderly, forced from their homes in the bitter cold".
"Hopefully this alert can be dealt with quickly and normality can resume for residents trying to get on with their daily lives," he said.
Those sentiments were echoed by Sinn FÃ©in's Elisha McCallion.
"Those behind this security alert have nothing to offer the local community," said Councillor McCallion.
"We don't need or want these type of incidents in the city."
A number of homes have been evacuated in Londonderry after the discovery of a suspicious device.