Slashdot Mirror
OpenProjects IRC Network Suffering DoS Attacks
Alowishus writes: "Open Projects Net, the IRC network which is home to Debian and other open source projects, has been suffering DoS attacks from a disgruntled customer of one of their server sponsors. Lilo, their sysadmin describes the attacks here, and asks for assistance." It's pretty terrible when a kid goes bananas and can damage the volunteer efforts of many people working really hard to create and support something so many of us use and enjoy. The sad part is that whoever is doing this feels self-righteous and justified in his actions, so nothing any of us say will
matter. I hope they catch him. DoS attacks just aren't cool, ever.
As admin of adams.openprojects.net, one of the servers that was DoS'd, I felt that I should give my own attempt at trying to find out exactly what was fueling his rage... he wouldn't tell me, or state any reason at all as to why he was doing this; he told me to ask someone else, who also didn't know.
This comment is mainly in response to one of the previous comments that basically scolded the slashdot crowd for not "understanding" the "pressure" this person was under, from VA Linux, apparently. How can we know what pressure he was under if he wont even tell us what happened? He's doing this practically as a "punishment", but we don't even know what we're being "punished" for.
I hope this ends with him being prosecuted, and I fully plan on submitting the 22MB log of eth0 traffic during the attack to the FBI and whoever else will be investigating it. I eventually plan on publishing the entire thing so everyone can see exactly how he formed these attacks. They were mostly the standard UDP floods, but the originating IPs are the interesting part.
With all the notes about how hard it is to prosecute this sort of activity, I really have to wonder... how long will it be before someone in a maddening situation like this cracks and puts a *real-life* hit on the luser?
:-). I just know I was sorely tempted at times when as an undergrad, high-school kids kept hacking into our school network. These were known to be local kids; hell, sometimes they would just walk in to the lab and shoulder-surf until they got a password, then sit down and log in. Our head admin chased one haX0r all the way out of the building and onto the dirt bike the kid had sitting outside the door...
:-).
I mean, sure, he's 31337 and all... but someone out there probably knows who he really is. And could be persuaded to go over there with a baseball bat and DoS his head.
Not that I advocate this at all
Anyway, I predict the development of Black ICE soon
It's a strange world -- let's keep it that way
I sponsor an openprojects server, vinge.openprojects.net, It's just for testing, we only have 5 users attached to it, yet this person felt it should be offline, so he sent over 100mb/sec down the pipe, slowing down the local backbone, (3 45mb DS3's at visi.com) and choking my poor ISP off for 15min.. that's not a long time, but to a modem customer who is surfing, or trying to get email, it's the end of the world. I just moved to this ISP recently, and I've allready got a bad rep with the admins. luckly I've known them for years, and they are personal friends, but it looks really bad when all i do is attract flooding, and DoS. they have enough problems with wu-ftpd scans, and netbios crap. I pay for the hosting, and the box out of pocket, so a few of my friends can get email and IRC, I ask myself, is it worth it? this is the closest i've come to saying no in the 2 years i've run nerp.net.
Open Projects Net: Denial of Service Attacks
Posted 7 Nov 2000 by lilo
Open Projects provides interactive facilities for coordination and support to groups and projects involved with open source. We run between 1,500 and 2,000 clients and are home to such projects as Debian GNU/Linux and Enlightenment. We've had our share of difficulties recently, but we're continuing on.
The past few weeks have been quite an experience. Last week one of our hubs on Open Projects started going up and down like a yoyo. I'd seen that behavior in this normally very reliable server in recent weeks and not thought much of it, since the company in question was in the process of moving its facilities and reliability issues do sometimes creep in during such moves. But we soon obtained a little bit more insight into the problem. After watching the server perform a loop-de-loop, I received a /MSG
from a rather peremptory and anonymous skript kiddie informing me that
if I didn't
permanently remove the sponsor's server from the network, he would kill
my home
ADSL
line and take down Open Projects until he got his way. It seems he
feels
the sponsor owes him money. I'm afraid I wasn't very polite in my
response.
Feeling that one can hardly allow psychotic delinquents to dictate
network
policy, I explained to him that while he might very well be able to take
down our network, he was not going to set policy, and specifically I
would
not entertain the notion of removing our sponsor's machine.
The last week has been interesting. Apparently this petulant child has something over 45Mbps to play with, and he's moderately competent with SYN attacks and so on. In various incidents throughout the week he packeted ISP's and universities and small companies to death to demonstrate his, uh, prowess with borrowed equipment. Currently he has proclaimed that he'll be taking down our network once a day for an hour until his wishes are granted. All I can say is that he's going to be doing it for a long time if that's the case; the heat death of the universe isn't due to arrive for some time.
Throughout this experience I have noticed it's very difficult to coordinate much of a response from ISPs and backbone providers. An unofficial contact at uu.net explained that we must notify his security people while an attack was taking place for them to have any chance of thwarting it. They thoughtfully provided him with an email address rather than a telephone number to give to us, explaining that this is a matter of policy. Perhaps they don't understand that packeting can affect services like email. Or perhaps they are simply extremely comfortable, their owners having cornered much of the backbone market after the last round of industry mergers. My employer's ISP was targeted, and so far the people at the ISP seem a little bewildered, though they're game to fight the good fight. Some folks with very nice bandwidth contributed a server today to see if we couldn't keep our hubbing working through an attack, and the skript kiddie seems to have gone after their routers, leaving very little in the way of evidence behind him as to his point of origin.
As a first, one of our admins contacted the FBI at our request. I'm not sure this will accomplish anything useful, but it's certainly worth a try. It is worth noting that, as a philosophical anarchist, I'm usually not inclined to bring in the muscle of a law enforcement agency to resolve such disputes, preferring to reason with the party or parties involved. But in cases where the problem user has learned his manners from repeated viewing of Robocop, well, there's not much one can do but consider the business to be a declaration of war.
At any rate, it seems to me that this otherwise very mundane set of attacks points to a long-standing problem with the Internet: Denial-of-service attackers have location indirection, but content services and users are left in plain sight as targets for their efforts. I'm hoping Corridors will helpful in dealing with this problem, though it's a fairly long-term project (and constantly in search of additional expertise to finish the design and begin the actual implementation). Meanwhile, we go on, attempting to devise kludges to improve the robustness of ircd in the face of all-out attack.
Any assistance from the readership in combatting problems which we have never experienced in quite this magnitude would be greatly appreciated.
Thanks to the Magenet people and Diane Bruce and F. John Rowan of the hybrid ircd project for their assistance. Thanks to the many users and admins of OPN, whose patience and support have been impressive. And thanks especially to VA Linux for their help and support; they've been real heroes and deserve a great deal of praise. And no, we're not going to delink their server, however many or few seconds we have to comply. ;)
--
314-15-9265
Isn't anyone else bothered by how it is always assumed that it is a kid thats making these attacks? I used to be a kid once, and didn't appreciate everything being blamed on my generation. Be realistic, people, it takes EXPERIENCE to become a true asshole; kids are amateurs at best... the REAL jerks are the seasoned veterans
Didn't you JUST say that DoS attacks weren't cool?
Hypocrite.
I've been involved with IRC in one way or another for about 7 years now. It's reasons like this that I do NOT run an IRC server anymore.
.jp servers, who we could never get the admins to fix. I'd also get people attacking my router directly, affecting thousands of customers, all over a silly IRC matter...
/dev/null. A wonderful discussion has gone on on the NANOG(North American Network Operators Group) mailing list, the past few weeks about this very problem. "IRC is stupid, don't make yourself a target" is something heard all too often. If people would just secure things now, when someone's attacking a web server, or something else of yours, you won't have that problem either. What if someone decided to DoS one of the major political party's web pages today, with the same types of floods? It's the same problem, but somehow this is worth investigating, but not if it's IRC? Yes, IRC isn't as philosophically important, but it's a very popular service, none the less...
Around 5 years ago, I ran toast.ne.us.dal.net, part of the DALnet IRC network (obviously). The bandwidth for it was generously donated by a local ISP, in exchange for borrowing some of my expertise from time to time. We only had a frame relay T1, but easily held more than 1000 users at a time.(Which was a record, for a short period) With popularity, attacks started coming.
The first thing that hit was SYN floods. Linux added the TCP cookies feature, which helped a bit. Then raw ICMP echo request floods, which caused us to get icmp blocked at our uplink, which hurt our customers, but was deemed worth while. Then when ICMP didn't work, people flooded the crap out of us with UDP. Then the Smurf attacks started. It came to a point that more often than not, during the evening, I was spending my time on the phone with our increasingly annoyed uplink getting things filtered and blocked.
In 1996, I moved to Illinois, and took the server with me. I started my own ISP on two T1's, and pretty immediately decided to pull the DALnet server, when the period of time that we're getting flooded exceeded the time we weren't. I then moved my IRC server to a much smaller network called NewNet. While the floods were much worse, it still was a perpetual annoyance that some brat in Israel decided he didn't like us, and would reguarly flood us from hacked
Then one day, the "script kiddies" discovered Wingate. Wingate is a highly useful Windows proxy system, that was unfortunately shipped for quite a long time in a highly insecure state. They had a telnet and SOCKS4 proxy sitting wide open, with no passwords necessary. One script out there would go scanning through cable modem and DSL netblocks, gather a list of a few thousand insecure wingates, and connecting them ALL to our network, using them to flood the crap out of us. No longer could we even ban naughty users, because they had thousands of hosts they could choose from.
One VERY frustrating day, I ended up writing a little tool to scan EVERY user who connected to our network, to see if they were actually connecting from an insecure proxy server. Worked wonders, but we had thousands of nasty e-mails from people asking why we were trying to hack them (by connecting to port 1080 then immediately disconnecting?). Much education was required, and many notices of "You're about to be scanned, disconnect if you don't want this to happen" were necessary to prevent some idiots with a firewall they didn't understand from flooding abuse@dragondata.com with nonsensical complaints about hack attempts.
Today, floods are much more sophisticated than the ones we saw 5 years ago. Current floods are completely legitimate TCP/IP packets, that look real. Not floods of SYN's, but real looking data, that you can't just slap a simple filter in to get rid of. Now, unless you're using a stateful firewall that can detect this sort of thing, you're pretty much screwed. (FreeBSD's ipfw system is now stateful, and works quite well for this sort of thing.)
Really, here are the major problems.
1) Network administrators don't secure their networks. They may secure their machines, but they let their routers blindly pass off spoofed packets, when it would be pretty easy in 99% of the cases to block packets with source addresses coming from a port that they don't belong in.
2) Any complaint to any abuse@ address that involves IRC seems to go into
3) It's nearly impossible to prosecute the people who do this. I've talked at great lengths with the FBI and other law enforcement agencies. While they sympathize, unless they have a huge dollar amount in damages they can show, there's little they can do.
4) The same companies and universities get hacked over and over again. I'd like to see someone sue one of them for negligance one of these times.
5) Stupid battles like this are really putting a drain on the IRC community. IRCD server software has pretty much gone untouched over the last few years, because any technically competant coders are busy coming up with proxy detectors and fighting floods, than writing code. There are things with IRC that could be done that would blow people away. But, I'm burned out. 7 years of fighting with people who need psychological help, because they do things like take down a huge network, instead of dealing with their issues in a constructive ways....
6) People take IRC too seriously. It's just for fun, people.
Kevin Day
According to Bill Machrone, the way to stop DoS attacks is:
1) Secure all servers
2) Liscense ISPs
3) Make spoofed packets illegal
4) Authenticate everything
5) Criminalize all scanning, including pings and probes
Now, would any of these solve openprojects.net's malaise? #1 wouldn't, because it's not their server which is launching the attack; #2 is a structural change which would take too long to implement (even if it's desirable); #3 is promising but would be an administrative nightmare; #4 we should be doing regardless; and #5 is perhaps a necessary evil.
The internet is fundamentally structurally vulnerable to DoS attacks. It's only a matter of pissing someone off and getting picked as a target. With the increasing politicization of everything on the net, the problems will only get worse.
-- Anne Marie