After leaving "Life" magazine, Smith joined the Magnum Photos agency in 1955. There he was commissioned by Stefan Lorant to produce a photographic profile of the city of Pittsburgh. The project was supposed to take him a month and to produce 100 images. It ended up occupying more than two years and producing 13,000 photographic negatives. The intended book was never delivered to Lorant, and Smith's obsessive work was bailed out by money from Magnum, causing strain between Smith and the photo-journalist collective.