Rob October 26, 2010 7:03 AM

That is to show that “security by obscurity” does not work.


Please, are there still people in the NSA who are not familiar with Joseph Needham’s work? No or few people who can read and speak Mandarin?

I remember an Israeli military advisor who told in a documentary that he once spoke to the US (CIA?) spook responsible for the Egyptian army. He could neither read nor speak Arabic.

tres malade October 26, 2010 7:45 AM

Seymour’s a great journalist but maybe a little weak in computer science. He writes:

‘He said that the crew of the EP-3E managed to erase the hard drive—“zeroed it out”—but did not destroy the hardware, which left data retrievable: “No one took a hammer.” … The Navy’s experts didn’t believe that China was capable of reverse-engineering the plane’s N.S.A.-supplied operating system, estimated at between thirty and fifty million lines of computer code,….’

This is fairly early in the piece. It sounds like he’s saying that they wiped the disk but didn’t destroy the hardware so that the Chinese were able to get to the code. How? Is this a matter of electronic techniques–that famous microscopy?

Moreover, how many minutes did the crew have? The plane would have been boarded as soon as it landed. Was the zero-out merrily spinning along while the pilot stalled the boarders?

In other words, it sounds like they didn’t zero-out the disk. Or am I missing something?

Seymour, if you’re listening, what do you mean?

BF Skinner October 26, 2010 7:48 AM

Good article. Hersh is the bomb (TSA/NSA Approved online saying!).

Often his articles strongly influence policy and public opinion…but with the issues so complex and the noise, from the election and various noise machines, how soon before it just becomes another unremembered drop in the noosphere?

BF Skinner October 26, 2010 8:04 AM

@tres malade ” it sounds like they didn’t zero-out the disk”

I’ve only heard of zeroize used in relation to clearing keys from cipher gear. It’s not to say they aren’t using it to describe harddrive clearing actions but most instructions makes reference to ‘clearing, purging, destruction’.

Some research and NIST say a single pass on an ATA drive is sufficient but a disk wipe even to DoD 5220.22-M standards hasn’t been acceptable to NSA for years. It’s degauss or nothing.

Still I heard data was recovered from harddrives recovered from the Columbia debries field.

Emergency action plans would have included hardware destruction. And yes we carried hammers to destroy cribs and circuit boards and other fragile electronics but a harddrive?

I’ve been told that there were thermite charges on the gear. Though thinking about it now lighting off thermite in a flying, pressurized, enclosed, crewed aircraft may be …not the right thing to do.

Still, pull the drive from it’s enclosure, open a freaking hatch and fling it into the ocean. I know they were just airdales but as navy surely they had a conceptual familiarity with ocean and had weighted bags to deep six the unburnable, unbangable stuff.

Jakub Narębski October 26, 2010 8:09 AM

It excellently shows why security by obscurity is a wrong thing to do. It is much easier to invalidate and change encryption keys, than replace reverse-engineered “sekrit” operating system.

tres malade October 26, 2010 9:29 AM

@ BF Skinner

What you seem to be saying is they zeroed out the encryption keys. But that implies that they then broke the encryption of the hard disk. Which is scarier–the Chinese hacking NSA military encryption and then reverse-engineering the underlying O/S; or going through a hard disk with one of those microscope things to recover all the data and then hacking the NSA military encryption.

BF Skinner October 26, 2010 9:40 AM

@tres ” implies that they then broke the encryption of the hard disk”

That would be scary but my take is that the keys were zero’d but the harddrives weren’t encrypted.

Anyone from NSA is welcome to correct me on this point.

colin October 26, 2010 9:45 AM

i don’t think it is a great article. it has some interesting facts that i wasn’t consciously aware of before but, if you really think about the e3 incident, and you remember china is a nation of >1B people… how do you imagine they’re not going to be able to reverse engineer some code? some of the quotes and the writing could be seen as jingoistic or racist. (why the crack about gunpowder? stupid.)

and pointing out that richard clarke said something about al queda before 9/11 is a weak argument. what’s more important is that, as the article’s author points out further along, technical critics pointed out serious flaws in clarke’s book.

tres malade October 26, 2010 10:00 AM

@ BF Skinner

It occurred to me afterward that it might have been as you say–that they zeroed the encryption keys used for data transmission but the hard disk wasn’t encrypted. Interesting issue.

Dave October 26, 2010 10:20 AM

Great article on ‘cyber security’ and what really goes on. Right now, I think that most of the government/private industry security processes run on inertia and not on common sense. No one individual wants to step up and rock the boat.

thecoldspy October 26, 2010 10:23 AM


The first rule of the cyber war “game” is make as much noise as you can so you can then reap as much money as will come.


About the “possibility” that xyz will happen at any given moment, thus in order to be prepared, we must obtain more data so we can then obtain more money. Thus the spreading of fear=money in the bank. We just might die tomorrow, so we better go out and try and prevent that from happening. Doesn’t mean we will die tomorrow, although some may surely die, but as long as we can claim the sky is falling on the net, and that a war is either imminent or already in progress, we can stave off from dying and get bloody rich at the same time. And, we don’t even need to know what we are talking about. All we need to do is use buzzwords, and yeah I read that story about Russians, or that other story about the Chinese, and yes, those evil doers will come after us next if we don’t ramp up another war on_______


In other words the military had long range goals to ramp up the fear they sell so well, and basically take over the command and control aspects of the net. Privately held companies in charge? Hah!! Never going to happen as long as Joe Jarhead has a chance to win one for the gipper.


Oh yes, he needs to read more emails, be on more social networks, and the internet, so he can protect…umm, wait for it… The CHILDREN… Yes by gumption the children need to be protected. If not for you, for them.


Why even write this? It is a known fact that Obama is NOT about transparency or open government or everyone singing Kumbaya. What he is about is obtaining more data on you, me, and everyone else.


Will Batman and Robin be able to escape their deadly trap? Stay tuned – same bat time – same bat channel..

What this is, is another long series of trumped up wars made possible by hyping the fear factor. What the reality is, is that the cyber industry needs that fear in order to grow. Without it, most of them would be out of work. So, hype it until everyone believes it, then make large suitcases of money, wash, rinse and repeat until you are a bloody billionaire.

That is the reality of this trumped up cyber war. It is only as real as you imagine it to be. Believe it or NOT..

Y.S. October 26, 2010 10:27 AM

That plane seizure..

I thought secret electronics like these came with self-destruct charges that can turn the whole thing into slag?

Is it that hard, to put thermite all over the stuff? Thermite is hard to ignite, so they’d just need to insert the initiators. It could probably be made quite safe.

Brandioch Conner October 26, 2010 10:35 AM


And yet not a single human casualty can be named in this “war” that we are “losing”.

“A great deal of money is at stake.”

Ding ding ding ding ding!!!
We have a WINNER!!!

This isn’t “war”. No one will die.

This is about getting more government funding. That’s all.

Jon October 26, 2010 11:16 AM

For me the most frightening quote from that article is, “(The Department of Homeland Security recently signed a memorandum with the Pentagon that gives the military authority to operate inside the United States in case of cyber attack.)”

Um, what? Signed a Memorandum!? Hello?!? And with talking about ‘using conventional forces against cyber attackers’, well, you just got martial law.

Have fun.


Brad Wheeler October 26, 2010 11:44 AM

It’s probably not a good sign that all of these quotes from “top administration officials” and military leadership would not sound at all out of place in an mid-90s hacker movie.

“Aircraft are literally falling out of the sky as a result of midair collisions across the country.” Oh, come on. Name one cyber attack that has had anything close to this dramatic of an effect. Even the Stuxnet stories that were tearing up the web a few weeks ago weren’t this dramatic, which is interesting given that that’s what government-sponsored digital warfare is likely to look like.

BF Skinner October 26, 2010 1:37 PM

@Bob “MAY have been fatalities ”

We discussed this here earlier in August (feature creep – an indexer/crossreferencer?)
It was pointed out that there was a lot of missing data and more study needed.

that said…perhaps defining what a cyber attack (necessitating response) is, is gonna be as hard as defining what a terrorist is.

They’ve made a start. The logic of getting dominance in this warfare theater will be to have already broken into the infrastructure of ANY nation that is or could be a threat, root and maintain access before formal hostilities break out…

Breaking into someones power distribution systems with the intent of having the abiltiy to disrupt them during a war would be an act of war itself wouldn’t it?

Perhaps this is McConnell’s meaning when he says we’re losing the cyberwar.

I think it would provoke the war they are trying to deter.

Interesting game Professor Falken the only winning move is not to play.

Davi Ottenheimer October 26, 2010 1:40 PM

My favorite part is when he dissects Howard Schmidt’s role as a “civilian” leader. I knew he was an ex LEO but did not realize he also was military.

At Black Hat this year I asked Jeff Moss why he offered the government a soapbox and gave a US General the keynote. That seemed very un-Black Hat. I also asked his Cyberwar panel whether the US already was ceding too much control to military leaders because of cyber war-drum beating.

The panel response was “no one is saying that” and “Howard Schmidt is a civilian”. I guess they were wrong on both counts. Will be interesting to see what’s changed by next year’s conference.

Jack October 26, 2010 1:41 PM

No one is going to get to the bottom of this issue. On one hand you have those like saying it’s all hype, then Clarke on the other hand saying the end is near. Then there are writers in between who only care about writing something people will read, especially read then come back for more. Then there are those who know something about the technology who will blow the whistle on the tiniest error to attract attention to their expertise, nevermind the issue itself and what could actually be true about it. Then there are those with political motives who don’t really care about the means, it’s just an end game to them. And of course the vendors who just want to sell more $20k appliances.

There is only one absolutely true take away from it all: Expect the unexpected.

mcb October 26, 2010 2:54 PM

I’m not normally paranoid (at least by the standards for this group) but I always figured the EP-3E incident was a cover for some other far more embarassing loss of the same information to the Chinese. If we really didn’t want the Chinese to have all this technology the pilot would have made for friendlier skies, bailed out at the last possible moment, and splashed the bird in deep water.

Brian W October 26, 2010 3:05 PM

I share some of the other commenters concerns about this accuracy of this. I have no insider information, but there seems to be a bunch of things that don’t add up.

1) Why do software “drivers” that allow for realtime decryption of Chinese communications also have the crypto keys for US communications? I could see such drivers disclosing what Chinese ciphers are compromised, but have no clue why they would also have US ciphers.

2) Why would a plane flying that close to China have on-board an OS which had the encryption keys to vital US communications? That’s just plain stupid. And why couldn’t they just supersede the crypto keys after the incident?

3) The only way I could see crypto keys not being cleared by zeroing the drive would be if they were physically encoded in the hardware (sort of like a TPM). Thus, with intact hardware the Chinese could theoretically decrypt any recovered data. Again, that seems like a horrible design choice for a system that will knowingly be put in a position where it could be compromised.

I also do not for one second believe that the Chinese “broke” US crypto. Perhaps they found an exploit or weakness that divulged the keys. But properly done crypto is impossible to brute force, by the NSA or anyone.

But I guess it is more politically palatable to claim that the Chinese possess god-like crypto-breaking capabilities than to admit that the NSA screwed up and designed a faulty system.

Richard Steven Hack October 26, 2010 5:37 PM

Here’s the bottom line: Military intelligence is an oxymoron.

Yes, all the stupid things mentioned probably occurred because – that’s what the military IS. A bunch of morons who like to blow stuff up and who are utterly incapable of doing anything that requires careful thought about consequences. They leave “consequences” to the civilian authority – and then bitch about it like McChrystal did.

Look, I was in the US Army from 1967 to 1970 including a year long stint in Vietnam. Read my lips – most military officers are absolute IDIOTS. And the longer they stay in and brown nose, the higher rank they get until you have…the Joint Chiefs of Staff who are absolute IDIOTS. Just look at the morons in JCS who were in charge during the Iraq war.

Giving these people ANY authority over ANYTHING that doesn’t involved direct military conflict with an opposing military in an actual physical battlefield is going to be an unmitigated disaster for everything and everyone in the vicinity of these morons. And in fact, even giving them charge on a physical battlefield results in disaster, as the Iraq and Afghan war Wikileaks documents demonstrate beyond all doubt.

Just look at Iraq and Afghanistan. Or ANY US military operation in the last century.

As for the Chinese, they have an estimated several hundred thousand young hackers in their university system. They are going to prove once again to people who really don’t get it the basic maxim of security: You can haz better security, you can haz worse security – but you can’t haz security.

For me, I was more interested in the trillion dollars of IP that can be stolen and sold to the Chinese. There’s more money to be made in that business than in tech support.

Also thanks for telling me – and the Chinese, if they didn’t already know – where the NSA has all their cyberspooks holed up. Now every hacker on the planet will be converging on that Baltimore airport annex. Think of the goodies and the reputation if you bust those guys! It will be more fun than Hack the Flag!

AC2 October 27, 2010 1:02 AM



…, thereby leaving the other 23 crew members to perish in a watery grave.. Or maybe you now have a better suggestion, but remember that he landed on an island 15 miles from the mainland…

From the article:


I’m sorry but I’m just unable to make the necessary leap of logic from the EP-3E incident to the prognosis of a full-blown ‘cyber war’ with China or another nation.

The article does go on to debunk the connection between the two and clearly distinguish espionage from war AND make the point that the whole thing is about le $s and control, but I doubt many people will read it too the bloody end…

AC2 October 27, 2010 1:11 AM

And frankly I don’t understand all the confusion over the zero-ising of the hard drive…

Its simply means to write 0’s to every byte on the drive. There are other more complex mechanisms, including one proposed by Bruce hisself, have a look at the manual for a tool from Western Digital that does this (apart from other things like backup etc) at the following link. Sec 18.3.2 is the one…


But as someone has pointed out above this is not considered to be a sufficiently robust approach..

RobertT October 27, 2010 5:07 AM

I have absolutely no knowledge of the EP-E3 systems involved, but I seem to remember that, not long after the event, the security community had a sudden renewed interest in an effect called memory data burnin.

zeor-ing the disk and crypto keys has little value if the circuit hardware RAM is in place and the attacker has the necessary skills to read the data that was previously stored in the memory. My guess would be that the EP-E3 systems used mainly SRAM, instead of DRAM as used in modern PC’s. SRAM is particularly susceptible to data burn-in by an effect called hot carrier injection. There are definitely plenty of Chinese semiconductor experts that are aware of this data burn-in effect, AND have the technical skills to tease the latent information out of the powered down chips.

Again I have no knowledge of the events but the careful wording of the article makes me wonder….

tres malade October 27, 2010 7:46 AM

@ AC2,

At the risk of being a little catty, I think we all know how a zero-out would work on the hard disk. The problem is elsewhere.

If the disk was zeroed-out, how did the Chinese find an O/S to reverse-engineer?

How long would a zero-out of a disk take? Did the crew have that much time before they were boarded?

Was the disk encrypted? If so, did the Chinese decrypt it–whether or not they had to use microscopy techniques to recover the possibly zeroed-out disk?

If the disk wasn’t encrypted, why not?

What was zeroed out–the encryption keys on encryption hardware?

Was the plane collecting information on hard disk for later off loading once it arrived at base or was it transmitting encrypted information as it worked (i.e. what encryption facilities did it need?)

EdT. October 27, 2010 8:44 AM

Uh… why didn’t they just set off a thermite charge as they were getting ready to deplane, and destroy the equipment that way? Don’t tell me they didn’t learn from the Tehran Embassy and USS Pueblo incidents?


mcb October 27, 2010 10:30 AM

@ AC2


…thereby leaving the other 23 crew members to perish in a watery grave. Or maybe you now have a better suggestion, but remember that he landed on an island 15 miles from the mainland…”

If the entire crew did not have parachutes then ditching the aircraft at sea would seem a better solution than landing at a Chinese air force base, if protecting the technology was the first priority. If the safety of his crew – under any and all circumstances – was his first concern then the pilot did a fine job.

David Thornley October 27, 2010 10:55 AM

As BF Skinner pointed out, what’s wrong with having sensitive Navy stuff being able to be pulled out, put in a weighted bag, and dumped overboard? It doesn’t always work (early in WWI, the German cruiser Magdeburg grounded, allowing the Russians to capture the code books because there was no deep water), but it has the virtue of being an option most of the places the USN operates.

One would expect that, if secret stuff is in a position where it might be captured, there would be at least one positive means of destroying or permanently losing it.

Geek Prophet October 27, 2010 1:12 PM


That wasn’t a “crack about gunpowder”. That was the author agreeing with you. He was mocking the claim that the Chinese couldn’t reverse engineer the OS, not agreeing with it.

Joe October 27, 2010 1:17 PM

I think that instead of writing zeros to the disk, which provides a uniform “foreground” which makes the background easier to figure out, the system should write random strings of 0’s and 1’s. Should be harder to regain the original data.

BF Skinner October 27, 2010 6:11 PM

Interesting link thanks

@EdT “…Pueblo”
We did learn from the Pueblo. Safety of crew is paramount. The only member who died during the assault was the comms officer trying to complete the EAP.

The lesson of the Pueblo could be either
a) don’t piss off paranoids with guns
b) start the EAP early.

Personnel safety is paramount. Remember it – It’s on the test.

AC2 October 28, 2010 1:17 AM

@ tres malade

Here’s one possible solution, based on the fact that just zeroising is not enough to erase data beyond all hope of recovery..

a. Disk not encrypted
b. OS on the disk
c. Disk zeroised but not physically destroyed (hammer/ degauss)
d. Chinese retrieve the executables for the OS from the disk and reverse engineer
e. Why disk not encrypted is a good question for which I have no answer
f. The discussion around deleting “encryption keys” is IMHO entirely a product of the discussion in the comments above and finds no mention in the article on the incident itself

tres malade October 28, 2010 4:16 AM

@ AC2

Merci bien.

I agree that’s one likely scenario.

The encryption keys stuff is from someone who seems to know. I.e. the term ‘zeroize’ is normally used for wiping encryption keys; it’s not normally used (as far as we know) for wiping disks and memory.

The basic problem at this point in the article is that as it’s written it just doesn’t make sense as to what happened. You have to concoct a scenario which more or less fits what Mr. Hersh wrote. ‘Twould be better if Mr. Hersh could check with his sources and then clarify what he means.

AC2 October 28, 2010 6:24 AM

@ tres malade


True, but the quote in the article says “zeroed it out” while talking about the disk…

Which IMHO just means a zero-fill (or maybe a one of the slightly more complex schemes) to erase data on a hard drive…

BF Skinner October 28, 2010 8:12 AM

@AC2 ” but the quote in the article says “zeroed it out” while talking about the disk… ”

Which I think is probably loosely applied terminology from Hersh’s source and not understood well enough for him to clarify.

We’re speculating here of course but we are informing the speculation with our experience.

To do even one pass write of 0’s on every sector of all the harddrives in the time they had doesn’t seem likely. HDs are getting bigger every year and then there’s this – OSs generally can’t wipe the system partition.

Every system partition drive I’ve ever had to wipe has to have been mounted as a slave drive on another device or booted to floopy drive or usb mounted OS.

Why weren’t the HD’s encrypted (positing that they weren’t on the plane and I think it likely they weren’t). And here we are assuming the HDs are COTS electromagnetic mechanical drives not solid-state or optical.

Here’s the thing. Even in a more hazardous enviornment the greatest concern remains protecting communication between endpoints. The endpoints are assumed to have sufficient protection (otherwise they don’t get to be endpoints). This is why SCIF on a Blackberry for Flag officers and SESs is such a worry.

The first thing to destroy in the EAP is the keymat. With OTAR I’m not sure how much keymat anyone is carrying any more probably just initial ignition, tactical I&A for the flight crew voice comms, maybe for their IFF transponders. The cipher hardware carrying these keys would be zero’d out.

The second thing is key pieces of cipher gear. Circuit boards, cribs and the like, flash memory, eproms.

Then signals sent and recieved. Then pubs and procedural manuals.

Where mass storage devices are in this I don’t know. I would have designed the the hd’s to be removable. (one for secure storage between use and the other to stick in a weight bag and sink).

The physical destruction of a HD is problamatic. They are really tough to get into, a strong arm and a sledge hammer takes a lot of time to break the casing. Multiply that by however many casings you have. Try it. I have. once you’ve got the disks out what do you do with them? A sufficiently paitent and funded advesary could recover data written to it.

While the E3 is a big plane with a crew of 24 there might not be room for a degausser. And if there is probably only one. Which becomes a choke point during destruction.

Here’s why I think it likely they weren’t encrypted inflight. Cost. Key management costs a lot. Each device would have to have a key, each key would have to accounted for through it’s life cycle (creation, use, escrow, change, destruction). Think in terms of thousands and tens of thousands of clients.

If OTAR doesn’t deliver these keys to the device with the HD then the crew is back to carrying keymat on board. And if you lose your key due to a system malfunction. the HD is now a brick. That has certain, unacceptable, mission impacting consequences.

HDs on PCs or servers on the secure networks (even TS/SCI circuits) are unencrypted. They run various flavors of COTS OSs (*nix and Windows). If the OS doesn’t support full disk encryption then it’s not going to be implemented. And that encryption has to satisfy NSA.

The security of that material is locally provided by the walls, intrusion alarms, vault doors, personnel suitability, camera’s and dvrs, guards making rounds and, possibily, watch to watch inventories.

mcb October 28, 2010 11:47 AM

@ BF Skinner

“Personnel safety is paramount. Remember it – It’s on the test.”

Isn’t ELINT a combat mission? If so, how is personnel safety reconciled with mission completion?

Clive Robinson October 28, 2010 12:30 PM

@ BF Skinner,


There are also the technical issues of HD encryption.

Without going into messy details it is a difficult problem for a number of reasons.

Often the simplest solution used is an Inline Media Encryptor (which the NSA make available these days) However these have big disadvantages as they are usually “effectivly one (master) key” in operation and usually have absolutly no knowledge of how data is stored on the disk above sector level.

Idealy any HD encryption should be as a consiquence of the application & the user ID and the OS & the File System in use as well as the individual sectors.

It can be done (I’ve done it) but it is a real pain to get right, oh and of course a certain well known and popular OS manufacture does not make sufficient information available (to mear mortals) in a manner to make it viable as a product (and believe me they will send a flock of legal vulture that makes Hitchcoks “the Birds” look tame in comparison if you do try…).

TED Vinson October 29, 2010 12:25 PM

@Richard Steven Hack

..wrote: “Here’s the bottom line: Military intelligence is an oxymoron.”
People who trot out this tired line usually do not know what they are talking about.

Thanks for your service in Vietnam. However, should you really be basing your judgments about the capabilities of military personnel on your >40 year old recollections? Maybe, just maybe, a few things have changed since then…

BTW: I’m sure FANX is not news to China or just about anyone else, including Google:

stvs October 29, 2010 1:18 PM

This article, like many of Hersh’s, is a mix of remarkable-if-true revelations and fundamental mistakes that cast doubt on the credibility of the whole.

Here Hersh displays a profound misunderstanding about the difference between privacy and security:

“It does remain far too easy for a sophisticated hacker to break into American networks. … People routinely open e-mails with infected attachments, allowing hackers to “enslave” their computers. … One solution is mandated encryption.”

This obviously makes no sense, as any exploit targeting my machine doesn’t care if it’s transmitted over a secure channel or not. In fact, encryption makes the problem harder simply because it’s harder for anyone to detect the malware en route.

Hersh follows this logical rat-hole for a few more paragraphs before bringing in Diffie to point out that encryption isn’t really the problem. Does Hersh understand Diffie’s point? He doesn’t appear to.

I’ve seen Hersh pull in jaw-dropping non sequiturs like this in other articles. They just make you shake your head and wonder if he makes the whole thing up as he goes. For example, Hersh claimed in a 2007 article the amazing fact that the Bush administration was assisting Lebanese/Saudi government-allied Al Qaeda forces in Lebanon based on an interview with Hezbollah’s Nasrallah. [] Wow! Except that for anyone with any knowledge of the region’s politics knows that this could only be a nonsensical fantasy. But never mind, events a few weeks later conspired to blow Hersh’s entire thesis out of the water when the Lebanese army clashed with their enemies, the extremist Fatah al-Islam that Hersh said the US was secretly helping. Here’s Michael Totten’s synopsis on Hersh’s embarrassing article. []

A track record like this and the cyber article’s rather glaring mistakes make me quite skeptical. Do we have independent confirmation of Hersh’s other revelations in this article?

Sam October 29, 2010 2:20 PM

reading the article reminds me of the saying “[the US] is afraid of its own shadow” as we say here in Syria.
The US military seems literally paranoid.

tres malade October 30, 2010 8:42 AM

@ stvs

Stvs raises three issues about Seymour Hersh (who is 73 years old):

  1. Is he an honorable journalist/honorable man?
  2. Is he a competent and professional journalist?
  3. Does he lack a good understanding of computing and has that lack of understanding influenced the quality of the article in question?

My personal view is that the answer is yes to all three questions.

stvs seems to be leveraging both a yes to 3 and a reference to events in Lebanon to give a resounding no to 1 and a fortiori to 2. I think stvs is wrong.

Hersh’s track record:

RobertT October 30, 2010 10:29 PM

Do you guys realize how irrelevant all this discussion about disk encryption and Key zeroizing is IF the Chinese attack focused on the system RAM and resister space of the processor?

If the complete operational state of the computer can be recovered, then this includes all the disk decryption and crypto activation keys.

The problem with special purpose computers (especially crypto hardware) is that they generally run only one specific program so the register data and program data gets easily burned into the RAM cells. Semiconductor effects like metal migration, NBTI and oxide charge trapping (causing MOS source / drain asymmetry) , can be used to determine the last stored value, for all memory cells in the system.

This is style of attack is WAY beyond the capabilities of script Kiddies BUT is it actually beyond the capabilities of a well funded / motivated State actor?

Nick P October 30, 2010 11:01 PM

@ RobertT

How much does the equipment and software needed for your attack cost? And can a regular electronic engineering major from college pull it off? Also, how much time on average does it take to extract, say, the registers, cache, and maybe first MB or 32 of RAM (likely key storage area)?

Clive Robinson October 31, 2010 12:17 AM

@ RobertT,

“Semiconductor effects like metal migration NBTI and oxide charge trapping (causing MOS source / drain asymmetry) , can be used to determine the last stored value, for all memory cells in the system.”


“BUT is it actually beyond the capabilities of a well funded / motivated State actor”

I don’t think it’s beyond the reach of some University students (or at least it didn’t used to be).

The last time I used a Scanning Electron Microscope (SEM) you could quite clearly measure the geometry of a chip surface.

Now I know we are several generations down from that point in time but I’m assuming lab equipment has atleast kept some kind of pace with it.

The problem of “burn in” on chips has as far as I’m aware been known reasonably well since the early 1980’s and there are ways to address the issue of,

“… they generally run only one specific program so the register dat and program data gets easily burned into the RAM cells.”

I mentioned it on this blog some years ago as “a snake eating it’s tail” that is the crypto key etc is not held directly in the computer but rather “as the shadow” between a number of bytes each of which is changed very rapidly.

The simplest way to think of the shadow is as “difference data” at the simplest the result of XORing two or more bytes together or the result of subtracting one value from another. Provided the difference between the bytes does not change when it is required the bytes can be of any value and changed as frequently as possible.

So over simply for easy visualisation let’s say you have two counters (Cnt1 and Cnt2) that both get incremented as an atomic operation. But one counter is always X more than the other. The secret value X can be calculated simply by finding the difference. But you never calculate it as a direct value, it is simply done as part of another processlike like Y += Cnt1, Y -= Cnt2; it has the same result on Y as X = Cnt1 – Cnt2, Y += X; but X is not actually stored in any RAM location.

Now if you consider a block cipher using Fiestel rounds you usually have the key expanded into sub keys for the individual rounds. Each sub key can be stored as a “shadow” and not as a real value. Further the shadow holding structure can be implemented as a number of circular buffers that rotate not just around each buffers RAM but also into any rounds. Further it is possible to design the rounds such that you can arbitarily inject a value in in one round and remove it again in another round (the consiquence of repeated XORing) so aditional truely random shadow data can be added and subtracted as “whitening” as a continuous process.

Provided your design ensures that the shadows are equally balenced across each and every bit in time then the burn in on each bit is the same.

The hard parts (which I’m not going to go into) are, working out how to build the shadows in a standalone system at boot up and how to securly implement the pointers required to make the buffers etc work.

However in the case of equipment that has a permanent power source like a battery the first issue is fairly easily obviated as it could have been in this particular case. Put simply when the equipment is powered the Keys etc could be injected into the equipment as shadows from a KeyMat fill gun, which never need leave the airbase. Likewise the fill gun can be loaded with shadows from a device that generates the shadows BUT never ever leaves the KeyMat holding facility. This sort of KeyMat handeling via fill gun used to be a quite standard part of key material managment and distrubution in forward and front line Commcens and is reasonably well understood.

RobertT October 31, 2010 1:33 AM

@Nick P
“How much does the equipment and software needed for your attack cost?”

At the RAM level: if the attack is possible than it will be relatively cheap to implement, say a few thousand dollars and a couple of man weeks developing specialized support hardware / software.

If you cannot extract the RAM data than there is no point trying to extract the processor registers.

Simplest case:
COT’s SRAM is used
No task startup location randomization was undertaken by the application.

Data Recovery:
1) RAM module removed from the system board ASAP and frozen (colder the better) Maintain this cold state during all testing. (requires thermo-stream) or a lot of liquid Nitrogen.

2) Disable any memory module PCB level initialization hardware (power on reset pulse) Hint: it connects to a pin called EN or something similar.

3) Remove memory broad PCB level chip power supply regulator (if any): replace with a bench supply with controllable turn-on / off voltage ramp rate. BTW if you don’t have such a supply than use a DC coupled Audio Amplifier as a power supply

4) Ramp the VDD turn-on at a rate slower than the chip internal power-on time constant (typ under 1usec). I’d try VDD ramp rates between 10usec and 1msec. (you’ll would need to perfect this technique on samples of the same modules, obtained some-how, get it wrong and you will erase the data)

When you achieve power-up without causing an on chip reset, than you simply read the data stored in the module.


The reason this simple technique works is that data burnin causes an offset to develop in what is otherwise a symmetrical balanced memory cell, this voltage offset causes the cell to powerup in the last state.

“Can a regular electronic engineering major from college pull it off”
Absolutely! (they may not understand why the technique works, but they an certainly carry out the tasks.

“Also, how much time on average does it take?”

There is no point trying to extract the processor data if the RAM data is not extractable. The extracting the RAM data should only take a few weeks regardless of the RAM size.

Extracting Processor data:

Far more complex:
1) Freeze processor chip ASAP
2) Buy identical processor chip from somewhere
3) Decap purchased chip and locate all critical internal Reg blocks and RAM blocks.
4) Identify the internal RAM / reg file reset circuits and disable ( probably requires (Focused Ion Beam) FIB access)

5) Identify processor power-up procedure that maintains data state (as above change VDD ramp rates)

6) write a program to dump the chip internal state after power-up (BTW look at the chip test modes because this is something the chip makers needs to do so it will already be supported in the Scan-chain.)

7) repeat steps 4 -6 on the real processor

It is likely that not all bits will be perfectly recoverable, so you might need to understand how the CRC blocks work and also look through the extracted code for likely sequences and correct unlikely code sequences where single bit errors are suspected.

So the most time is likely to be required to transform the almost correct into a fully functioning program, this is a task any software developer familiar with assembly code can do. How long…depends on how many errors and how obvious the errors are.

Remember: many parts of the RAM / processor will be completely unrecoverable, because the data they contain is always changing, you need to simply accept this and plug in likely values for things like stack pointers Areg …etc

RobertT October 31, 2010 2:38 AM

Preventing data burn-in is certainly possible but was it done? that is the question….

I agree with your proposed crypto key storage methods (as a difference of two counters) there are similar methods where the secure data is XOR’ed with a clock which automatically flips the cell data and the read cell. Unfortunately even this difference technique is not fool proof because the difference is usually resolved by with an XOR gate, meaning that the offsets occur on this gate. The correct technique is to embed the key in the difference of two or more changing sub keys where the crypto calculation resolves the difference. Unfortunately not everyone designs systems securely….

I’ve just seen far to many instances, in commercial special purpose equipment, where simple data burn-in precautions were not taken resulting in keys being recovered from old circuit boards.

Typically stopping data burn-in of the executable program requires that the start location of the program index registers is somehow randomized…all relatively easy to do, but was it done?

BTW: If you can see the surface if the chip (using a SEM) than when the chip is powered up the state of all the SRAM bits can be easily determined by a simile technique called “voltage contrast microscopy”

If the SEM method is is too difficult, another technique called OBRICH or LIVA can be used. These can be combined with IDDQ measurements from a scanning laser to allow data recovery of the internal chip RAM contents. This gives you and XY map of the bit data which needs to be transformed into the actual RAM address/data map by reversing engineering the row / col decoders.

Clive Robinson October 31, 2010 5:07 AM

@ Nick P,


There is more than one type of attack.

A quick bit of ancient (in computing terms) history. there where many different types of storage media tried before the use of integrated circuit memory. Two that you may know about are “ferrite core” memory and “CRT Storage tube”. Both suffered from “residual data” issues that fell into two proad categories,

1, Non permanent retention.
2, Permanent deformation.

In the case of ferrite core memory the first was partial magnetisation of the core and was an integral part of the design. In effect you used the core like a tiny soft iron magnet by placing a large current pulse in one direction it became a weak magnet pointing in one axsis or it’s opposit depending on which direction the write current pulse went around the core. The direction of magnetisation could be read out later via the sense wire mechanism. However If you repeatedly wrote the same direction to the core you would eventualy turn it from being a very soft magnet to an almost permanent magnet in that direction at which point the core could not be pulled back by the ordinary write wire mechanism, and the data state was effectivly permanently writen in.

Likewise Storage tube CRT’s these held an illuminated spot on the screen by the use of what amounts to static charge and low level flooding by electron beam. They had a tiny grid inside the tube just behind the phoshor screen that could hold discreet packets of charge almost like miniture capacitors that leaked their charge to the phosphor screen and kept it illuminated after the main electron beam had passed. In order to maintain the bright spot the charge had to be “topped up” by a flood beam that provided about the same charge into the grid as was leaking out to the phosphor.

Even if you turned the flood beam off the charge would remain for a very long time and in a dark room the phosphor could still be seen vaguly emmiting light many minutes and sometimes hours after the power was removed from the tube and could with care be recorded with a camera with it’s shutter deliberatly held open for several seconds (and yes such screen recording cameras where quite common in electronics research labs up into the 1980’s). However there was a downside to CRT storage tubes the suffered very easily to a from of permanent “burn in” all CRT tubes suffer from often called “screen burn”. That is the grid or the phosphor would become permanently damaged and could be seen from then onwards as a change in brightness on the screen as the electron beam went over the effected areas.

Now the important thing to remember is to some extent all memory devices and many transducers suffer from both these two problems. For instance incandescent (filament) light bulbs suffer from metal migration, run them on DC and they will (on average) fail a lot sooner than if you powered them from AC. Silicon chips suffer from both problems as well.

Silicon memory storage elements generaly come in one of two conceptual forms,

1, A latch made from two NOR gates with crossed over feedback (ie output of gat A to input of gate B).
2, An insulation gate Field Effect Transistor (FET) with a capacitor between the gate and source on which charge is held.

Due to silicon “real estate” issues the first is not realy used much these days.

So the first question that arises of just how long can the charge be held on the gate capacitor?

The answer depends on the design but in the case of EPROM / EEPROM / FLASH the answer appears to be tens of years. So the residual charge time on a RAM chip can be a very long time. For instance back in the early 1980’s I had designed a small Z80 CPU card and it used Dynamic RAM (DRAM) which requires to be refreshed every few milliseconds.

The third prototype had a problem it appeared to work fine during testing but would go wrong after about 3/4 of an hour for no apparent reason. The fault was eventually found to be a break in the PCB track on the refresh circuit. Now even alowing for the fact that ordinary memory writes effectivly did a memory refresh we found that the data appeared to remain good for atleast ten minutes…

Also Remember this was without any spescialised equipment and at around 25 degrees C or ~77F (it was summer and in the UK we did not have “AirCon” back in the 80’s you were lucky if you had a window you could actually see out of due to that frosted safety glass with wires in and where I was working for security reasons they had even put a “white wash” paint coat on the glass…)

So (1) Non permanent retention can be a lot longer than you would think even without the use of Liquid Nitrogen. And RoberT has detailed some of the ways it can be got at.

But (1) presupposes that the Chinese where ready and waiting with all the bit’s required to take advantage of the “time window” of the aircraft landing. Does this sound likley if it was an accident that brought the plane down?

Which kind of suggests that (1) was not the route they used (I don’t know if CO2 fire extinquishers would lengthen the time significantly).

But what of (2) Permanent deformation, well it’s not just light bulbs that suffer from metal migration sillicon chips do as well. As a simple rule of thumb all normal conductors have IR losses which means they are potential fuses, that is at some current the conductor will actually melt. Long before this point however the metal will actually “go with the flow” of current how much depends on a number of things, one of which is the total charge transfere (effectivly current multipled by duration).

However the net effect is under a DC current the conducter will get thinner at on end, and fatter at the other, this is effectivly permanant deformation as sillicon chips as a rule don’t like having AC applied to the power rails ( at the very least it kind of messes things up internaly as it goes through the memory cell bias zero crossing point which can and has be used as an attack vector).

Importantly at the point where the conductor is getting thinner the resistance and therefore IR loss goes up. And there are a number of ways this can be seen/exploited (the simplest for ordinary conductors is by the slight increase in temprature at that point seen as an infrared hot spot). Temprature is not the only method and I suspect it is not practical with modern memory chips.

However as RobertT noted metal migration in the drain and source circuits etc is not the only permanent damage a “static” data state inflicts on a memory cell within a memory chip.

One method that has be used by “hackers” in the past on the “supposadly erased” SRAM in “Satellite” and “cable” set top boxes and the like is to power the chip up slowly at different tempratures and read the data out. You can do this over and over and average out random state changes and find the state a memory cell prefers to be in (which might be indicative of the state it was almost permanently in or just the chip layout).

Again I don’t know how well this will work with modern memory chips but this is not the only way it can be done. One simple enhancment to this simple method was to add a small AC waveform to the DC power supply (google “A-D Ditthering” for why it works).

If it works sufficiently reliably this is a very very cheap in equipment terms attack, requiring little physical complexity thus could be done by an undergraduate (it is however time expensive).

Now the questions that follow are what are the other methods open to exploitation? and are they known to the US and/or the Chinese?

If you have a number of people activly designing memory chips and activly researching the physics of the materials involved what other methods can they come up with for reading the (2) Permanent Deformation.

It is known that in times past the NSA was the leading light in semiconductor research, and some of their employees / associates left to form their own companies to further develop technology for them. Super Computers are perhaps the most well known area but telecomms as well (like CELP voice codecs etc).

Importantly it is suspected that the NSA are still the worlds experts on data storage simply due to their own rapacious needs (Never Say Anything, but store everything you see and hear ;).

But you also need to remember that you find a large number of Chinese and ethnic Chinese researchers all over the globe in organisations researching into both security and chip development. And further that both China’s have used such people in the past to “steal secrets” at all levels (think about where the expression “Chinese knock off” comes from and how “China got the bomb”).

So it might well be that the Chinese have the same or more knowledge in this area with regards to the US and other WASP nations.

All of which muddies the water quite a lot as to how China might have got the data together.

stvs October 31, 2010 5:53 AM

@ tres malade, My personal view is that the answer is yes to all three questions.

That may be my personal view as well, but personal views are irrelevant if not supported by facts.

I presented a few examples of egregious, fundamental mistakes in Hersh’s articles—including the recent cyber article—that should make us skeptical. It would not be inappropriate to ask if Hersh allowed himself to be misled by some of his sources in these instances, or to ask for independent confirmation of his remarkable revelations.

Please address facts, not your personal views.

RobertT October 31, 2010 7:45 AM

@Nick P
The following is high level physics and technical gobbledygook but just for accuracy reasons:
Differences in Mosfet Drain / Source region are not usually due to Metal migration. Actually the main mechanism’s are related to hot carrier injection and hydrogen ion contamination of the Si/SiO2 drain interface layer. The gate oxide injection occurs at the Drain end of the MOS channel rather than the Source end (due to higher electric fields around the Drain and the presence of GIDL current).

When a normal Lateral MOSfet is fabricated the Drain and Source are symmetric about the Gate, so the behavior is the exactly same if the Source and Drain functions are reversed. However once a device operated for a while, a semi-permanent charge trapping occurs in the gate oxide (for modern devices about 10nm to 30nm from the drain side of the gate edge). Since this charge is now asymmetric it alters the device threshold (Vt) so the Gate Source and Gate Drain Vt’s differ. This effect is utilized in some Flash memories that use thick Nitride gates to “permanently” trap this charge at one end or the other.

This effect can be used to discover the typical state of any SRAM cells without even powering up the SRAM cell, because the PASS transistors develop this data dependent Source/Drain charge asymmetry. (typical SRAM has 2 pass transistors and two back to back inverters )

Clearly this level of on chip device analysis is outside your average undergraduate skill set, but well known by any memory R&D engineer, and btw today practically all semiconductor memories come from Asia.

As for detecting burn-in state by metal migration this is in principle possible using a technique known as Seebeck effect imaging, Ive never seen it actually done, but….

Similar forensic techniques exist to discover the data on a disk drive that has been erased, even if the drive is first zeroed and than has random data written over the disk.

Bottom line: if you want to keep your data secure NEVER underestimate your adversary.

John November 1, 2010 8:21 AM

The article can be summed up by this statement:


Leave a comment



Sidebar photo of Bruce Schneier by Joe MacInnis.