Residents of Princes Drive in Weymouth, whose gardens back on to the mainline to London Waterloo, said trains stop with their engines running for up to five hours on Saturdays.
They said their weekend noise nightmare started last year.
Great Western Railway (GWR) said new procedures had now been put in place to minimise the time engines run.
Peter Yarwood, who has lived on Princes Drive for 15 years, said: "We quite enjoy the trains passing but to have one parked outside your house for five hours on a Saturday is not very nice."
As well as the "constant droning noise" he said he could not open his windows or sit in the garden because of the fumes.
Resident Wayne Wilcox agreed: "The noise is bad enough but the smell is horrendous."
The trains wait outside Weymouth Station from about midday on Saturdays until 17:00 on some occasions, residents said.
In a letter to GWR in May, Mark Whitchelo said the fumes were "filling" his garden.
He said he and his wife Diane were concerned about pollution caused by the fumes.
The couple contacted MP Richard Drax, who said GWR was "doing all they can to solve the problem".
A GWR spokesman said: "Generally engines need to run to provide and maintain the air supply for the doors and brakes systems to work.
"Safety checks need to be carried out before the train can go in to service and these checks require the engine to be running."
But he admitted it should not happen for five hours and said new procedures had since been introduced at Weymouth which involved both engines being shut down once checks had been carried out.
Noise, fumes and the "horrendous smell" of idling diesel trains are blighting the lives of people in a seaside town.