Lafitte is believed to have been born either in Basque-France or the French colony of Saint-Domingue. By 1805, he was operating a warehouse in New Orleans to help disperse the goods smuggled by his brother Pierre Lafitte. The United States government passed the Embargo Act of 1807, so the Lafittes moved their operations to an island in Barataria Bay, Louisiana. By 1810, their new port had become very successful; the Lafittes had a profitable smuggling operation and also started to engage in piracy.