
Search I,Cringely:

The Pulpit
The Pulpit

<< [ The Plot Thickens ]   |  The Cat is Out of the Bag  |   [ Have Another Cup? ] >>

Weekly Column

The Cat is Out of the Bag: Why DDoS May be Even Worse Than You Think. A LOT Worse

Status: [CLOSED]
By Robert X. Cringely

Editor's Note: The following contains mostly unedited e-mails from readers, so don't waste your time sending e-mails about grammatical errors to Bob. Thanks.

I have the best, the smartest, and the most cynical readers anywhere, and they came through big time when I asked for more details on Distributed Denial of Service. This is the last column I will devote to this subject, which the rest of the journalistic world has already abandoned. And abandoned too soon you'll see as you read some of the comments below. There is one important correction I need to make to last week's work. I blamed Solaris for the problem, primarily because I was hearing from folks at Sun that their software was the source of difficulty. This week, I have had it proved to my satisfaction that as much trouble was posed by systems running Windows NT, and that the underlying operating system makes little difference in how these attacks are instigated.

Thanks to the dozens of people who sent information. Below, I have tried to include most of the ideas presented to me by a number of people. There was substantial duplication of ideas. If I didn't include your work here, it is because I don't want to make the column so long that people won't read all the way to the end. PLEASE read all the way to the end, especially the quite long final section. Get ready to be scared.

Eric Rachner wrote:
"Securing the bulk of Internet end systems is not a realistic solution. However, there is a realistic solution to the DDoS problem and it has been available from the IETF in RFC 2267 for about two years now. From a civic point of view there are a lot of safe practices which ISPs need to abide by in order to maintain the overall health of the net infrastructure. Effective spam prevention is a well-known example, and one that ISPs have ample incentive to implement. If tech journalism was more tech and less journalism, RFC 2267-style filtering would be undergoing an awareness groundswell right about now. While cynics may argue that lax ISPs will always be numerous enough to sustain attacks like these, I expect proposals forthcoming from Federal committees to be even less realistic and probably more expensive."

Elizabeth Olson wrote:
"I could be wrong, but I think part of why the method of attack wasn't officially revealed by the FBI was the same classic security mistake that companies make over and over again - attempting security through obscurity. 'If everyone knew how to do it, everyone would do it!' is often the tagline to this sentiment, despite the fact that anyone who knows anything at all about security knows that the only way to get problems fixed is to expose them. Lists such as Bugtraq exist just for this purpose. In fact, I read through recent posts to Bugtraq to see what they had to say about the attacks. There was a great synopsis of some of the tools used to generate the attacks as well as methods for stopping them posted February 10.

"While most people outside security circles (i.e. anyone who doesn't read Bugtraq) wouldn't have read this, the media usually doesn't pick up on the technical aspects of such things. When a refinery blows up we hear it's because a valve failed and little more. The same goes here and the engineers who are responsible for fixing it are well informed as to the causes and fixes for the problem. I don't see any conspiracy, just a lack of interest by the common guy. This was the same problem that was had when smurf attacks abounded and it took a very, very long time to get even large ISPs to fix their networks such that they couldn't be used as relays. System administrators are slowly wising up and the community at large is starting to realize that in fact no man is an island and a greater mechanism for cooperation in such matters is required.

"The fact remains, the FBI didn't say how it was done for the same dumb reasons they probably wouldn't say anything about anything else - silence gives them a feeling of security they can't attain otherwise, because they are basically powerless against this. It is totally in the hands of people who operate networks to fix it. There will always be vandals on the Internet and the FBI catching one of them won't do a damned thing. In this case it's the unfortunate fact that the victims are responsible for preventing the actions of the perpetrator, and that's just life."

Jay Kangel wrote:
"At some point one of these hacking events is going to cost someone who can hire lots of lawyers with real money. At that point the victim, or the victim's insurance company, will want to sue for damages. The actual hacker will likely have little or no money. Even if the victim wins such a suit the damages cannot be recovered. The deep pockets are the owners of the zombie machines. Is it negligence if a machine owner does not promptly install security patches and, as a result, hackers take over the machine? I don't know..."

Bob Lewis wrote:
"Maybe the government, itslef, is launching the attacks? Nah. Well it's probably a couple of — holes with an attitude about e-commerce or possibly people who were trying to short-sell tech stocks, but if you are in the mood for a conspiracy theory I would advance the following. It wasn't the government per se, as in an order from Clinton, but consider that the FBI has been trying to get telecom and ISPs to install the equipment and pipes to peel off all backbone traffic and send it to them for surveillance for about two years and all these pesky privacy advocates (as well as the ISPs that would have to PAY for this equipment) have been railing against it. The FBI is only trying to do what the FSB, successor to the KGB, is doing in Russia. So post-cold war, the FBI=KGB. Now consider if you're a real patriotic gung-ho FBI guy and your team stands to gain considerable money and clout from this kind of disruption. It's not too tough to set this up so it freaks everyone out, yet doesn't leave any real fingerprints.... You didn't do any real damage and any that was done could be considered acceptable losses given your righteous mission. A couple of days later you have a couple hundred million in your budget, mandate to set up ten regional response centers, etc., etc. With your real objective accomplished, you let the boys chase ghosts and maybe eventually find someone along the line who spit on the sidewalk so you get some kind of conviction. Disclaimer: I DO have the site, but you heard it here first. J"

Finally, Charlie Demerjian wrote, and wrote, and wrote: "....A part-time employee (soon to be full time) of mine was the first person to characterize and post the info on these attacks. The institution he works for was shut down for almost two weeks last August while they were figuring out what was going on and how to stop it. While I am sure this was not the first DdoS attack, I cannot find a published report of anything sooner. Basically he was, and still is, on the front line. Needless to say he spent much of the last week talking to the FBI for a number of reasons. I have spent a LONG time talking to him about what happened, how it happened, and what the future holds, so here are my answers to your questions....

"First a bit of history on DdoS attacks. The attacks that happened last August were simply a new usage of the DdoS tools. The first time I saw them was back in '93 or so. I was working nights at a hospital at a major midwestern university with nothing to do all night. While sleep was an option I live at night so I stayed up and played on the Net. I almost always had an IRC window up no matter what else I was doing at the time. If you are familiar with IRC, you know there are bots on most established channels to keep the peace, provide rules to newbies and other boring housekeeping tasks. Occasionally you need to remove or ban a person from a channel due to unsocial behavior, personal hygiene, or other things. Attack scripts developed and defense scripts soon followed and the cat and mouse game began. Not too long after that a reasonable stalemate was achieved and new ways of removing people were needed. The solution that no one found a way around was ping flooding.

"What ping flooding amounted to was pinging someone so often that the ping/useful data ration on his line would be so low that his machine would time out and drop off the net. Because most computers at that time were on a dialup connection, a bot on a dedicated UNIX box at a large university could easily remove someone from the net. A simple IRC command of /flood was an almost universal way to remove anyone. This was fine for 99.9% of users, but those on high speed lines posed a harder problem. The solution came with the rise of linked bots.

"The linked bots basically started as a way to keep control of a channel. If you have three bots talking to each other, if one is attacked the other two immediately go after the hapless attacker. This soon led to "bot net." Bot net was formed by 15-20 channels of like-minded people (*cough* pirates *cough*). Any bot linked in this fashion would pass a /kill command to the other 100-200 bots on the net and they would all flood the target. It could be called by anyone "authorized" on any of the participating channels and was rarely abused because any of a thousand people could call it. If you screwed around with it, you would almost certainly taste it soon. It was a nuclear deterrent situation. It was also remarkably effective. I cannot remember a single person who withstood it. I know I had three or so bot on a T-3 line back then and that alone was almost enough to remove anyone by itself.

"That was the last I heard of the technique until last August when my friend was attacked. He told me about the 'new' attack that hit his place of work and I chuckled and that was about it. I forgot about it until Yahoo got hit. When it became obvious that this was the next thing in hacking, it started a lively discussion in the little circle of geeks I travel in.... Here is what I know.

"1) How are these attacks made? Basically they are incredibly easy to pull off. There are attack programs readily downloadable from most 'security' sites. All you need to do is get the programs and find a bunch of host machines to use it on. The hosts can be almost anything and if you don't know how to compromise a computer look at those same security sites. They have pre-rolled root kits for almost ANY OS.

"While the DDoS tools have many variants, they almost all follow the same general outline. It goes something like this:

"A) A 'master' box is hacked. While they have been generally reported to be fast machines, they really don't have to be. They don't do much other than signal a start and stop.

"B) You hack a bunch of 'slave' machines. The more the merrier, and the faster the line they are on, the better. The speed of the machine is not all that important - almost any modern P-II machine can saturate a 100 Base-T line - so filling a T-3 or an OC-12 is no problem. Line speed is key here. Also there is a brisk trade for compromised machines. If you can find ten of them yourself (not hard) you can easily trade that for 100 more. If you spend a week preparing, it is easy to get as many slaves as you want.

"C) You give the master a list of slaves, a target, and a time. If you have half a brain you cover your tracks and set the thing to remove itself.

"D) At the set time, the master signals all the slaves and they start ending data to the target. While the target may not consume a single CPU cycle looking at these packets, the lines leading to the servers will almost certainly become so clogged that nothing useful gets through. There are variants that will go after the server targeted, but they are not necessary, clogging the lines is enough.

"E) The target sits and waits because it can't get any data in. It may be able to send data out at full rate, but without anyone being able to request that data, not much happens. To the outside world it looks like the site is down. Please note that NO amount of patches or fixes can do a damn thing about this. It's not the OS that's attacked, but the pipes leading up to it. A Ferrari doesn't do you much good after a four foot snowstorm, especially if the streets are not plowed.

"F) The people running the servers under attack now have to trace back 1000 machines pinging them and notify the owners that their boxes are causing problems. This is compounded by the fact that most people don't know that their computers are participating in the attack. To give you an idea of the task that stopping this requires, try the following exercise. Pick any four 8-bit numbers. Now try to contact the owner of that IP address. Remember time is limited and, oh yeah, your main Internet connection is down. Have fun. Repeat 999 times. Unplugging your line does not stop the attack and still leave you down. As soon as you plug back in you pick up where you left off. Basically you just sit there until the attacker gets bored and stops.

"The end result is that almost any antisocial 14 year-old with a fifth grade reading level and a not-too-short attantion span can take Yahoo, or anyone else, down....

"What particular vulnerabilities were exploited? Basically none. I know of several types of boxes that can be used as masters or slaves. You mentioned Solaris, and my friend turned a Redhat 6.0 box over to the FBI Wednesday. Almost any UNIX will do, and I am sure the software has been compiled for everything under the sun (no pun intended). There are three major variants of the DdoS tools and countless others that have been modified to use a different port, different packets for signals, etc.

"These attacks do not exploit any particular property but can be made to use ANY existing vulnerability. I am sure that as each new hole pops up in an OS, it will be added to the easily downloadable scripts. Just think, when the first Win2000 hole is found, in the week it takes MS to patch it you can use the 17 million Win2K boxes for the next wave. Sigh.

"What can be done to avoid future attacks? In my opinion, nothing. The cat is out of the bag......"

Comments from the Tribe

Status: [CLOSED] read all comments (0)