Kharroubi started playing football at the age of 6 in his neighborhood club "US Vénissieux". At the age of 8, he joined the Olympique Lyon academy, and stayed with the club until the age of 15. He would then join the academy at Grenoble Foot for another 4 years, reaching the semi-finals of the Coupe Gambardella, before leaving to Brazil at age 19 to sign with Bangu. After one season, he was loaned out to Série A side Esporte Clube Vitória and at the end of the season he would come back to France to join Valenciennes FC in the French Championnat National. In his first season, he played 33 games and helped the team finish first in the tables gaining promotion to Ligue 2. In his second season, injuries limited him to just 10 games (and 1 goal) but his team still finished first in the standings, gaining promotion to Ligue 1. In his first season in the top flight, Kharroubi again managed just 10 games. Injuries again limited him to just 3 games in his second season in Ligue 1.