Data mining


Electronic snooping by the state may safeguard liberty—and also threaten it

See article

Readers' comments



Alan McCrindle

Another delusionSome points that I think that this story missed 1. The record of data mining and modelling in our current financial crisis - ratings agencies used "sophisticated" financial models based on historical data mining to assess the risk of sub- prime loans - they rated them AAA and actually believed that they were correct - they even provided software that the banks could use to get an idea of the rating they would get from the type of CDO they were constructing.Blind Freddy should have been able to work out that many of the sub-prime loans had no chance of being paid back. It is called common sense. Computer programs don't contain common sense unless it has been programmed in. People who believe that the world is 100% rational and can be explained by simplistic models that include assumptions like the "hidden hand" of the market are deluded fundamentalist simpletonsTake away points: Models are only as good as their assumptions. What happened in the past is not a predictor of what will happen in the future. We live in a non-linear world, reductionists and simple linear thinkers go home.2. Computer models such as these make us less resilient and less safe.These models create and "arms race" that simply lead to the "terrorists" developing more subtle and sophisticated techniques to evade our models. Unless we understand this and keep up with this arms race we get left behind and find ourselves in a more vulnerable position. The problem is that once we develop models we think that they are keeping us safe and let down our guards. Technology and "sophistication" is not always best. We can end up deskilling ourselves in the basics.To use a medical analogy, hospitals now hook patients up to machines that measure and monitor things like blood pressure, body temperature, heart rate etc. The doctors no longer "get their hands dirty" doing this basic work. But there are many unrecognised consequences of this new sophisticated technology. Much of medicine is based on trust and the placebo - this trust is built by the "bedside manner" of the doctor and touching the patient is very much part of this process. In addition, we can tell a lot by touching a patient that a machine can't measure. Measuring a pulse by hand provides all sorts of information other than the pulse rate. Because modern medicine has lost this art it is now in denial that it exists.3. Treating symptoms does not remove the cause of a problem. Unfortunately we live in a world where the reductive scientific world view runs the shop. This world view only believes what can be measured objectively. Unfortunately people like suicide bombers are not objective rationalists. They are driven by "hidden, non rational" causes. Just as treating the symptoms of chronic disease will not remove the cause of the disease, so attempting to stop terrorism by attempting to stop people committing the acts will not stop terrorism. We can see this deluded short term, symptomatic level, thinking in operation with our current economic melt down and the proposed plans to solve the problem - exactly where in Paulson's original plans were there any measures to deal with this causes of the problem? And even in the modified version being debated now where is there recognition of the causes other than in the question of inflated salaries?We live in a world dominated by fundamentalists - either short term thinkers and short term politicians who understand the world using simplistic linear reductionist models or eternity reductionists who are in search of purity and perfection.


It makes no sense to compare WW2 with the "war on terror". WW2 was a war between states armed to the teeth wich caused massive losses of lives and property. Al Qaeda is essentially a criminal organisation which has killed far less people than die every year in car accidents on American roads. Of course it has to be addressed, but rationally, without losing sight of the bigger picture.

Extreme measures that might perhaps have been justified to defeat the Axis powers have a far too low benefit to cost ratio to make sense as a solution for what is essentially a policing problem. The risk that we in the West might have to live under a totalitarian islamist caliphate is zero; the risk that we might have to live under a nearly totalitarian state of our own making because of our lack of perspective is substantially higher.


The problem, Logicool, is that people flow through various ideas in their life, and if someone happens to become radicalized and wishes death upon innocent people for a period of time, does not mean he will remain that way forever. Perhaps the situation has changed, the anger has abated or has been redirected to something more tangible. Do you think that the man who has cleared his heart of vengeance will still be allowed to travel? No. The problem we have is once the accused are placed on the list, they will most likely never be taken off no matter what they do.We all go through times of turmoil and confusion, particularly teenagers and particularly people who see first hand the effects of collateral damage. I would not so easily trust these lists. Error is ubiquitous in this world and these lists are becoming considered infallible by the people who run them. Their word is becoming the law.

Nut Head

René Girard, a famous French anthropologist, developed the theory of ‘mimetic doubling.' The theory basically says this: the more two antagonists fight against each other, the more they become like one another. The mutual fascination of rivals leads to an unconscious ‘copying’ (mimesis) of each other: they become mirrors. Such is happening now. As the 'liberal' West fights against 'illiberal' Islamic radicals, the West becomes more 'illiberal' and Islamic radicals, by the very definition of the paradox, become more 'liberal.' Such a thing happened to Rome as it fought against the outside 'barbarians' with lower moral values. Odd how history repeats itself, eh?

Jose Carpintero

In my experience with data mining for private companies, it all lead to a lot of money for consultants, very little profit to justify the whole exercise, and a lot of junk mail for uninterested "potential consumers".

Translating it to the government world, I can see it all resulting in a lot of money for IT consultants, very little improvement in security, and a lot of harrasment for innocent "potential terrorists".

Remember that all the junk mail that you get in your box is sent by a computer program which says that your profile fits quite well and you are going to be very interested in the junk.


Data mining can be useful (and potentially dangerous to civil liberties) but it usually has to be seeded or directed by intelligence gathered by people on the ground. Dragnets and keyword searches are useless except in the cases where it is done forensically after the fact. So, I'm not sure that the purpose of the system is to prevent attacks as much as to rapidly respond to an attack (and perhaps prevent subsequent attacks). Of course it's a lot easier to get money when you claim to be able to prevent attacks.

Random Commentator

So, USA started information revolution and now killed it. Digital data are so easy to intercept that people will soon turn back to paper and cash. You will not be harassed by police because you googled for fertilizer for your grass lawn, which turns to be also material for chemical attack. You also bought electronic toy bear to your son, which might be turned into a fuse.Mass surveillance is also perfect opportunity to steal valuable information from companies or material to blackmail politicians and public figures. Society must be aware that this random and mass surveillance in information age is costly and dangerous. I hope it is curtailed sooner rather than later, before major scandals erupt.


Did I misread something? Where is Echelon project? In 1999, Economist reported "The Echelon project, a grid of listening posts linked to super-computers operated by Britain and by America’s National Security Agency, is reported to be able to detect key words in every fax, e-mail and telephone call made within the European Union." Ironically, this project has worked all these nine years and the magazine gets concerned now. Is it more serious now as the agency has started to crunch its own citizens' data?

Yankee Doodle

Random Commentator wrote: "Privacy right is human right. Its constitutional right of not showing your political and other views to the state."

Where is this Human Rights written, and by what definition? If you will not offer these to us we must then conclude that you WISH that this was a Human Right. If it is a right as you claim, then please, where is it written and how is it enforced?

Vino Comitatus

The free market and technology innovation will eventually provide a solution to the problem of government intrusion, it alwys does. There is a company in the U.S., called Vadium Technology, Inc., ( that has found a way to take the One-Time Pad Cipher in make work in the computer world. The OTP was created in 1917 by AT&T, and is the only cipher ever to be proven permanently unbreakable. That is right both he GCHQ and the U.S. NSA cannot break it. Instead of expecting government to police itself, everyone should take responsibility for thier own privacy, liberty and freedom.

Yankee Doodle


Human rights are always been diminished when wars rage. Human rights laws explicitly recognize this necessity. This loss is another among the many reasons to avoid wars.

Look at WWII, if the Allies had followed human rights laws and the Geneva Conventions every European classroom today would be adorned with swastikas. Instead, a decision to fight the Nazis with Nazi-like tactics was made. Was it wrong?

Random Commentator

Thousands of innocent people were denied flights, entry to U.S. or were questioned, even although they did nothing wrong. Mass surveillance is disturbing to people, it is not invisible data mining, and the cost to society is growing.

Yankee Doodle

Universal human rights do not yet include privacy rights. Some nations have constitutional limits upon privacy, but many do not. There is no universal standard for universal privacy rights, even if such a universal right were claimed.

The US Constitution offers privacy rights, but like all fundamental human rights, in times of war (like now) these may legally be reduced. The EU lacks constitutional rights, but each member has assured privacy rights, but these are not universal standards anywhere else (or even within other EU members). The UN Declaration of Human Rights does not assure anyone anywhere privacy. In the end, it matters where we live, and what the laws are where our messages travel to.


What if a potential terror group would use passenger pigeons and regular mail to communicate?! Not to fast but totally under the radar.


Let's look at the reverse of main thread of the discussion here. Would it be better for the police and authorities to focus their attention on all devout muslims for fear they are extremists or only the ones that have flown to Peshawar, posted to jihadist web sites, etc.? Which way is better for most Muslims?

Isn't narrowing down suspects what police and intelligence agents have always done? They're just trying to automate it. Right?

However, I aggree with Jose that the most like result of this effort will be vast amounts of data that no-one is really able to make much sense of.

Free Man_2

The tag line for the story is "Electronic snooping by the state may safeguard liberty and also threaten it",however I do not see in the story any examples of how these methods safeguard liberty.They may, on certain rare occasions, safeguards the lives of a small number of individuals.However this is decidedly not the same a safeguarding liberty."He who would trade liberty for some temporary security, deserves neither liberty nor security."Benjamin Franklin.


Well done to the economist for raising this important issue! One of the most useful things you can do with your browser is enable it to make random background web site visits - help to flood the data analysers with meaningless data.

Random Commentator

Privacy right is human right. Its constitutional right of not showing your political and other views to the state. Ultimately, privacy is also important to preserve democracy from abusive governments and lower officials. History teaches what happens when government is given excessive power to spy its citizens. Government unavoidably abuses the power. This brings fall of government and its country, whose people didn't give sharp slap into eavesdropping ear.Recent examples of abuse is Germany using anti-terrorist law to tackle tax evasion. Some councils in Great Britain apparently used power of spying for terrorists to search for people whose dogs sh*t the streets.

aidan clarke

Our intuitive conceptualisations of privacy have been left in the dust by the in this information age. A one way and secret gathering of information is indeed illiberal, but what is illiberal about public access to information about actions that I publicly undertake, especially if I am aware of this access?At most it would just lead to social mores becoming less two-faced, by closing the gap between what we say and what we do.


Latest blog posts - All times are GMT
World Cup predictions: Scoring ourselves
Graphic detail July 12th, 19:03
Car technology: Turbocharged
Schumpeter July 12th, 9:00
Gun laws: Prayers and pistols
Democracy in America July 11th, 17:43


Products and events



