The "defeat device" allows cars to pass lab testing even though they actually emit 40 times the emissions standard.
The US Environmental Protection Agency (EPA) has been taking a more aggressive stance on car pollution and violations of the country's Clean Air Act.
The recall and fines could cost the carmaker more than $18bn (£11.5bn).
It affects 2009-14 Jettas, Beetles, Audi A3s and Golfs and 2014-15 Passats.
"Using a defeat device in cars to evade clean air standards is illegal and a threat to public health," said Cynthia Giles, assistant administrator for the Office of Enforcement and Compliance Assurance.
The illegal system allowed cars to detect when they were undergoing smog emission test and lowered the rate of pollution. Those emission controls were then turned off during ordinary use.
The state of California which assisted in the investigation has also issued a notice of violation to Volkswagen.
In 2014 the EPA fined Korean automakers Hyundai Motor and Kia Motors $300m for misrepresenting the fuel economy in 1.2 million of their cars. That settlement is the highest to date.
German carmaker Volkswagen has been ordered by US regulators to recall half a million cars because of a device that disguises pollution levels.