Into the 20th century, the diminishing importance of the Chicago & Northwestern Railroad line through Manchester slowed train traffic and cut into the town's expansion, causing many to leave the town. This slide continued into the Great Depression as the line lay idle and more residents were forced to close their businesses and move elsewhere to find work. Despite road contact finally being made through Manchester in the late 1930s, little remained to induce people to linger and the few remaining residents began to die off or move away, with many relocating to nearby De Smet or Huron. Having lost many of its residents and its primary means of attracting more, Manchester's population steadily dwindled until a core population of no more than 100 residents remained, with most operating farms and ranches outside the town limits. The CNW railroad officially announced its intended permanent abandonment of the railroad line in 1986, but later sold the tracks to the Dakota, Minnesota and Eastern Railroad. By 2003, less than a dozen structures (including two operating businesses) remained standing on the original Manchester town plot along US-14 and 425th Avenue.