Smith was best known for running a clean program and having a high graduation rate, with 96.6% of his athletes receiving their degrees. While at North Carolina, Smith helped promote desegregation by recruiting the university's first African-American scholarship basketball player, Charlie Scott, and pushing for equal treatment for African Americans by local businesses. Smith coached and worked with numerous people at North Carolina who achieved notable success in basketball, as players, coaches, or both. Smith retired in 1997, saying that he was not able to give the team the same level of enthusiasm that he had given it for years. After retiring, Smith used his influence to help various charitable ventures and liberal political activities, but in his latter years he suffered from advanced dementia and ceased most public activities.