From a Cajun family, Desormeaux grew up in a rural farming area located a few miles outside Maurice, Louisiana. His parents are Brenda and Harris Desormeaux. "I was born and raised here", his mother said. "We speak French and are proud of it. My dad is the one who bought Kent his first saddle. There's always been a strong connection to this countryside." His brother, Keith, older by three years, is a race horse trainer. They have four younger siblings, Kristie, Kelli, Kalen and Kip. His mother explained the names: "Keith was named for my little brother John Keith, who drowned. I just liked the name Kent and then we just decided to go with the K's." Desormeaux was a member of the local 4-H club, and was first exposed to race-riding at age 12. Keith Desormeaux explained that, "The bush tracks were all around us, and our dad decided he might want to delve into horse racing and bought a bush track Downs ... We lived in an agricultural area but we weren't farmers. Even before we got into racing, we all had horses to ride growing up."