Vitter was born on November 17, 1961 in New Orleans, Louisiana. He studied at Harvard University, at Magdalen College, Oxford, and at Tulane University. He is married to Wendy Baldwin. They have four children.