Justice first produced tracks together for Musclorvision's "Hits Up to You", a concept compilation where tracks were designed to sound like they were meant for the Eurovision song contest. The compilation included two of their songs, and a track by Augé's alter-ego Microloisir. Justice first came to prominence through its 2003 remix of Simian's track "Never Be Alone", created for a Paris college radio station's remix contest. The remix resulted in the group being signed to Ed Banger (who released it) and became a hit in clubs and on the Internet. The song was re-released in 2004 on DJ Hell's International DeeJay Gigolo Records and eventually given a commercial release in the UK in the summer of 2006 under the title "We Are Your Friends" on Ten Records, a sub-label of Virgin Records.