growing up in poverty in and on the fringes of new orleans ' notorious storyville red-light district , armstrong came early to hard work , to privation , to exposure to the seamy underside of urban life .