After graduating from William Henry Harrison High School in 1976, he attended Indiana State University-Evansville (now the University of Southern Indiana) where he received a bachelor's degree in sociology, and became a member of Sigma Tau Gamma. Ellsworth worked in the paint and hardware department at Sears while in school to pay for his education. He later received a master's degree in criminology from Indiana State University. Rose-Hulman Institute of Technology also gave him an honorary doctorate of humane letters at their 2008 commencement.