Wolfe lived in Barrington, Illinois, a suburb of Chicago, with his wife Rosemary, where they raised four children. Wolfe also has three granddaughters. The Wolfes moved to Peoria, Illinois in 2013. Wolfe underwent double bypass surgery on April 24, 2010. Wolfe also underwent cataract surgery on his right eye in early 2013. Wolfe's wife, Rosemary, died on December 14, 2013, after a series of illnesses, including Alzheimer's disease. Wolfe said, "There was a time when she did not remember my name or that we were married, but she still remembered that she loved me."