When his father died in late 1889, 16-year-old Edmondson refused to continue to tirelessly work on the plantation and relocated to Nashville for work. He got a good job working at the expansive new Nashville, Chattanooga and St. Louis Railroad shops. After an injury sustained at the Railway shops in 1907, Edmondson took a job as a custodian at the Nashville Women's Hospital, where he worked until the hospital closed in 1931. During this time, the Edmondson family officially resided in Nashville as his mother, Jane, was the last to leave the Compton Plantation. By age 39, his wages at Women's Hospital allowed him to buy a modest home and spacious lot in Edgehill, a segregated neighborhood in Nashville's Vanderbilt-Belmont area. Edmondson never married and he shared the home with his mother and sister, as well as, occasionally, other siblings, nieces, and nephews. With Jane's death in 1922, Edmonson's sister, Sarah, soon became the woman of the house. It was said she would often hold family barbecues that featured much laughter and storytelling. When the hospital closed in 1931, Edmondson picked up work at various part-time jobs and in his leisure, he could be found in his backyard sculpting and selling his vegetables.