Slashdot Mirror
MS SQL Server Worm Wreaking Havoc
defile writes "Since about midnight EST almost every host on the internet has been receiving a 376 byte UDP payload on port ms-sql-m (1434) from a random infected server. Reports of some hosts receiving 10 per minute or more. internetpulse.net is reporting UUNet and Internap are being hit very hard. This is the cause of major connectivity problems being experienced worldwide. It is believed this worm leverages a vulnerability published
in June 2002. Several core routers have taken to blocking port 1434 outright.
If you run Microsoft SQL Server, make sure the public internet can't access it. If you manage a gateway, consider dropping UDP packets sent to port 1434." bani adds "This has effectively disabled 5 of the 13 root nameservers."
In South Korea internet services were shut down nationwide for hours on Saturday, the country's Yonhap news agency reported.
It said the shutdown was triggered by "apparent cyber terror committed by hackers".
http://news.bbc.co.uk/1/hi/technology/2693925.stm
Where I work we ended up with quiet the excitement. Around 1am I lost connectivity on my DSL modem at my house.. and I just figured something was up with the DSL so I fooled around with that for a while.... but then I realized the data light on the hub for the DSL modem was blinking a WHOLE lot and nothing else on the hub was (ie broadcasts were coming through)... I couldn't ping our core router, nothing... YIKES! So I hiked into work... only to find that 3 machines had been compromised. A co-lo we have, and some other ones. Nothing bad mind you.. easy to fix.. install Service Pack, and then firewall the ports out.. but still.... it was interesting.. I walked into the server room and was greated with a ton of orange lights (that are normally just blinking!) That thing can really cook out the damage!
Someone really has carefully crafted this worm to try to bring down the net.. and what better time then on a Saturday morning when all admins are away and not planing to work the next day!
Waking up at 2AM after falling asleep at work on a Friday evening, to be greeted by a wall full of router racks lit up like a wall-shaped christmas tree is a sobering experience indeed. Needless to say I've been working since then to apply appropriate firewall rules accross our network to block port 1434. Once this blows over, it's time to start some real PostgreSQL advocacy..
What does this worm rank compared to other DDOS in the past?
I was very surprised to discover both AP and CNN beat Slashdot to this story.
Very disappointing.
Timely is as important as accurate SlashEditors. Many of us look to you when big events occur...
Especially considering this all began about 8 hours ago!
e3 :: blogging the wireless freenet
I've been watching this havoc unfold all night as well. I wonder how long it's going to take for the entire problem to clear. Most sites that were previously unaccessible are for me are now, except some of our own. Makes me wonder if something else is going on in these datacenters.
No, firewalls are for use as your needs require.
I, for instance allow no incoming, but don't restrict outgoing. It's not a huge corporation, it's a R + D lab, where the overhead and hassle I'd cause by restricting outbound traffic would stiffle the lab users productivity. Still, I added the block to that specfic port in the slim chance that an internal box was infected (lord knows how) that it would be a localised problem, not contributing.
I don't think you should tell people what firewall rules they should be running.
Get your own free personal location tracker
No reason? Really? What about distributed servers taking to a central database? Desktop software that queries a remote database? Remote administration of a remote database? All legitimate reasons.
No point in having a router that can't sustain max-traffic on the network it's put on...
What if your campus get slashdotted ? Kinda boring if the router shutsdown because of legit traffic
My guess is that some MSCP caught panic when he saw the load on the mssql-server and pulled the plug...
It's happened to me... (and he wasn't even MSCP just vanilla dumb...)
Gr.... All the more reason to run a host firewall on every machine.
Need a Linux consultant in New Orleans?
Is this thing directly targetting root/tld servers? Is the worm doing dns lookups as opposed to just picking an ipaddr? Is it the PTR servers which are being hammered by loggers doing reverse lookups?
Did someone jump to a bad conclusion based on ping stats?
I don't know if anyone else has had the same problem, but xxx@msn.com email addresses seem to not be working on Hotmail. I doubt they're related, but has anyone else had the same problem, and is this likely to be the cause? By the way, xxx@hotmail.com accounts work fine.
"Tier 1 backbones are reporting a bad night: routing instabilities, one major dropped most of its peering for a while, the volume from this triggers the Cisco netflow switching bug and is causing routers to lock up at places, etc."
About half of the sources I've seen have been either .edu sites or sites in other countries which belong to colleges (ualberta.ca, etc.). Is there some sinister corellation here? Perhaps colleges get free MS-ware, and let the students run the networks?
I want to delete my account but Slashdot doesn't allow it.
given also this previous slashdot story, the root servers must join and sue microsoft for DDOS attacks against them.
Windows clients send TOO much shit to any dns - check your dnscache log to see that. Don't have a dnscache? Bad! You're flooding your preferred DNS server with a shitload of useless or meaningless queries.
Looks like they have read some websites some years ago and then decided to steal words like "domain", thus confusing a nt-domain and a REAL domain name. The rest is pure mess because nt-domains are queried with DNS. Pretty crappy isn't it?
Look at that (dnscache log):
@400000003e329b973170f1bc tx 0 33 _kerberos._tcp.dc._msdcs.[mydomain]. . 97010201
@400000003e329b973874c81c tx 0 33 _kerberos._tcp.dc._msdcs.[mydomain]. . 97010201 97010101
@400000003e329b981c3f8394 tx 0 33 _kerberos._tcp.dc._msdcs.[mydomain]. . 97010101
this is a laptop trying to find a network share on the server (which is called server2000.[mydomain].it). It is querying [mydomain], not [mydomain].it as I set up the laptop (default domain, network identification). Imagine if I did not have a dnscache but set up all PCs to use an external dns server....
-- There are two kind of sysadmins: Paranoids and Losers. (adapted from D. Bach)
If a unix vulnerability was ever exploited to the levels that this sql one or nimbda or sircam were, I'm sure one of you AC's would let us know!
It's amazing how many people just don't feal they have to upgrade their machines. Im stil getting nimda hits. The sql exploit is using a vulnerability 6mos old!
Show's you the real vulnerability is the image the MS has palmed off on the public for 20 years! With our system you don't need to worry about good administration! It just works and works and works! Why pay for an admin when you can by MS Win-X?
-- Many men would appreciate a woman's mind more if they could fondle it
you guys just dont get it, conspiracy theorists that tries to get anything out of the numbers....!
personally i think it was george w. bush, in a maniacal attempt to force feed us with laws to protect the internet from terrorists!
Put together a website listing all the IP address that sent you port 1434/udp 376 traffic.
My firewall blocked 167 of these requests before we lost our upstream connection (our co-lo ISP gets its bandwidth from uu.net, which was hosed).
Write a short script to get whois/admin info, then send automated email to management pointing out the stupidity of running an unpatched server months after the fix was available, and the stupidity of having it available unfiltered to the internet.
Point out that there are many smart, unemployed tech people who wouldn't allow this kind of stupidity to occur.
(Yes, I know MS patches sometimes break other things, and you need to test them before deploying them. But it has been many months.)
I have argued for many years that people tend to get the idea that a firewall is some kind of +8 amulet of protection they just strap on which will protect them from pretty much anything.
However there are real benefits to using firewalls and NAT boxes. Unfortunately there are some members of the IESG who are confused on this point but thats because they are blinkered by the end-to-end dogma. I'll note here that Steve Bellovin, the new security AD knows a thing or two about firewalls.
There are actually two end-to-end principles. Applied to networking it meant put the intelligence at the ends, not in the middle of a communication. This was applied to security to mean the same thing.
End-to-end is appropriate to the design of network protocols, it is inappropriate as a guide to operational security. Many protocols are not designed securely, most protocol implementations have flaws.
Another dogma that is inappropriate to operational security is the 'security through obscurity' trope. A design that relies on security through obscurity is broken. This does not mean that operators should divulge all the details of their operations to attackers in the hope this will improve security, it will not. Argument of this type was used to block the introduction of shadow passwords on UNIX for years after the vulnerability to dictionary attacks was widely known and being exploited by attackers.
A firewall and NAT box provides a significant degree of security at low cost. NAT provides a means of concealing the internal structure of the network. This does not eliminate the possibility of attack but raises the bar significantly. If you are running a site that is considered attractive to hackers a technology that weeds out the knob turners and dimmer script kiddies has value.
What we need to move to is security in depth, recognizing that design security and operational security are different and that both are important.
Looking for an Information Security student project suggestion?
Try http://dotcrimeManifesto.com/
Almost certainly the People's Liberation Army's Cyberwarfare division running a test by using an old exploit.
South Korea was also hit.
The PLA is probably just waiting for the US to be fully committed in Iraq before they let North Korea attack South Korea, Japan, the Phillipines and Alaska with missiles as a diversion so that they can enslave Free China.
I slapped a line on our access list in our BGP routers this morning at around 8:30 A.M. Even though our firewall was blocking this port, figured it would be better to block in silicon rather than at the O/S level. In almost 2 hours, we have recieved over 190,000 packets from this wurm. I have a feeling its going to get a lot worse before it gets better
I wouldn't say firewalls make people lazy; it's more a problem of people not understanding security.
These people are just as likely to say things like "I'm 3DES encrypting my data, so there's no way anybody can read it", because they fail to understand the meaning of statements like "cracking 3DES is computationally infeasible". When you try to explain to them that their webserver and applications are much more likely to be their weakness than their encryption algorithm, they give you blank looks and mutter about the Computerworld article that said 3DES is "unbreakable encryption". It's not a problem with 3DES (or any strong algorithm); it's a problem with people not understanding that any security measure can be negated by poor design in other parts of their architecture.
It's the same thing with firewalls. Only the unknowledgable would drop in a firewall and then go off to the bar to celebrate their newly "secure" network. That doesn't mean that the firewall is useless; it is still a crucial tool for securing one's network. The problem is the people who have no idea how to use the tool properly, and no concept of what a real-world attack actually looks like.
I work for an ISP and I just got home from work where we had to deal with this madness. It was absoultely horrible people. We got word from UUNET that it is port 1434/udp traffic and they are adding that to their egress filters. We just blocked 1434/udp altogether, at least initially.
We have many many colocated customers, many of whom run msql. This issue is horrible in that it is causing massive packet loss and when packets do get through the latency is around 500ms and up and that is for an all ethernet network segment. Our core router was getting slammed and cpu utilization would hang out at around 100%.
When we started unplugging switches from the routers, traffic would return to normal. We then pinpointed it down to all of our colo customers and disconnected just the sql servers from the network. Effing pain in the ass though.
Goddamned MS and their crappy no-password-requirement for the sql admin user and the moron admins who don't patch their system. Are people this trusting of MS that their servers are safe and/or this stupid they just don't apply patches until they get screwed?
Whatever, I am soooo tired... g'night
ZERO ZERO ONE ZERO ONE ZERO ONE ONE! Just brushing up for my next big invention: Ethernet over Voice (EoV)
Whoever puts a database outside a firewall
24,432 fuckwits have done so, counting the hits on my firewall. 1 hit on port 1434 yesterday, 0 on thursday.
Wait, there are some dups, it seems that each machine hits the same addresses over and over again, about once every 4 to 12 minutes. grep|awk|sort|uniq gives 11,901 unique IP addresses in my firewall logs.
Quickly scanning a statistical sampling of machines which have probed my IP space, I see that most of them are wide open to the internet. Ports 137/139, 25, 1029, etc. are all available, and 3 of the 11 show BackOrifice on port 31337.
I have a friend (oracle expert) over trying to set up a vulnerable MS Sqweal server so we can study the worms actions on an isolated test network. I want to see which addresses does it scan, rate of repetition, and other things, since the code is pretty simple and just hashes the addresses (low cyclical rate) over and over again. I've also learned some new bad Vlamsk (dutch) language today.
I've got a packet that might crash vulnerable MsSqueal server processes using the same buffer overflow technique. Could be a good return packet to send to scanning machines to get them to shut up until the admins get around to patching/rebooting their fucked windoze machines.
But first I will test it on my own machines, I really don't believe in affecting other's machines on the internet, even if the owners are fuckwits. But after yet another microshit worm fucking things up for everyone else, I've moved my limit closer to their processes.
the AC
I'm also waiting for the first few variants with better IP address scanning routines, which will be much more virulent. Monday will be a *fun* day
Hemos is like...sci-fi fans;he thinks technology is cool, but he hasn't bothered to understand the science it's based on
Was nice to see one coming from 207.46.196.109 which is activex.microsoft.com - used by m$ mediaplayer for codec downloads etc (it connects there first, then to codecs.m$.com or something)..
Interesting ports on activex.microsoft.com (207.46.196.109):
Port State Service
1434/udp open ms-sql-m
What do you want to own today?
... I wonder if evil-doers might be mining the Microsoft patch libraries, looking for exploits that already have fixes, but depending upon the cluelessness of Microsoft site admins to fail to implement them...
Why go to all the trouble to invent a problem, when there is a large population of targets and a database of vulnerabilities?
The problem with logging is that it is usless unless you actually review the logs. This rarely happens until after a site has been compromised.
Much more useful is to have the firewall connected up to a 24x7 monitoring, or better management service like Counterpane, VeriSign or whatever.
Over time I expect that cost of high end firewalls to drop significantly. I have two firewalls at home, neither cost more than $200 and they are both pretty adequate for my needs. So why does an enterprise setup cost $80K rather than $4K or so?
Looking for an Information Security student project suggestion?
Try http://dotcrimeManifesto.com/
Starting around the same time, www.whitehouse.net began receiving about 100 times the normal requests for the home page and its associated graphics. Most of the offending hosts are in China thought at least a few aren't. So far, there are at least 1000 distinct addresses spread accross their entire IP space that reloaded the page at least 30 times.
I have no direct evidence this is related to the worm, but it begs coincidence.
www.whitehouse.net is a privately-owned parody of the US White House web site.
Source samples with counts include:
3302 61.171.37.209
2443 218.17.216.111
2037 218.4.128.50
1962 218.25.204.219
1527 61.187.169.160
1336 61.131.48.222
1183 218.58.69.26
1079 68.37.179.107
Moderating "-1, Disagree" is simple censorship. Have the guts to post your opinion.
I went to buy groceries this morning and was told that the networks of all the major Canadian banks are down except TD. Plus I haven't been able to connect to the toronto star all morning. Now I'm hungry and searching the couch cushions for change to buy myself lunch and a newspaper. It's scary how reliant on the availability of network services i am. Might not be too bad of an idea to stash some cash under the matress just in case something much much worse ever happens.
My network got hit hard this morning. The article claims 10 packets per minute. We were getting 10 packets in about 1 nano second. It sent our firewall to a load average of 10+ and brought our entire network (inbound and outbound traffic) to a halt. We found a single Windows host causing all the problems _behind_ our firewall. After disconnecting it all was well again. Thank you MS.
Buffer overflows as a security hole aren't only a Microsoft problem -- although you would think they could afford better code reviews -- they are an almost universal C/C++ problem.
First, using fixed-size buffers for strings (and other arrays) seem almost to be encouraged by the language design, or at least by common practice.
Second, strings (and other arrays) unfortunately do not have a size inherently associated with them in the language, and null-terminated strings can be slow to check for length.
Third, the stack layout of typical C/C++ implementations makes it *possible* to overwrite the return address. Some other programming languages I have used had implementations with the return address below the local variables, making it essentially impossible to overwrite.
But then, years ago, nobody ever seemed to think about security issues in language design.
... is that our Corporate IT has *outsourced* all control of our firewalls (to a company which recently filed chapter 11, if I recall), and so can't update them on the fly...
And, on top of this, our "corporate IT security" just sent out an email that some of their *internal* machines were infected (so obviously *something* was accessable through the firewall) and now we who are connected to corporate via a T1 must apply the patches. So much for the firewall.
This also happened with Code Red two years ago. Big panic, everyone patching their systems, because corporate had holes in the firewall.
Yet, we have our own firewall to a customer site (which we've managed on our own for years, and which corporate now wants to take over) which we have *never* been infected via. Go figure.
Not saying that we shouldn't have been up on it, but we have noone dedicated to IT Security (funny, since we do DOD work) in our building, and we are all so swamped with other stuff we rarely have the time to keep up with it.
At my *last* job, however, we setup a new box and immediately port-scanned it... knew what every service was on the box, and if we didn't, closed it down. And that *wasn't* DOD... e-commerce. And we kept on top of patches.
So... you credit card number was *really* safe at my old job... but our nation's secrets may not be at the new job.
Go figure.
More from The Globe and Mail
e ws /front/RTGAM/20030125/wintern/Front/homeBN/breakin gnews
:-)
http://www.theglobeandmail.com/servlet/ArticleN
I especially like the nickname somebody gave it: "SQ Hell"
Funny how the site www.internettrafficreport.com is being slashdotted right now. In the last 5 min alone, the global traffic index went from 85 to 65, apparently a new wave of attacks as the worm discovers new ground. My 5-domain webserver hasnt received a packet yet, but Im keeping my eye on it. Glad to be using Postgres with its ports blocked from the Internet.
Holy cow! Israel is completely down according to the site.. all routers with 100% packet loss.
"Give orange me give eat orange me eat orange give me eat orange give me you." -Nim Chimpsky
A link to this thread has hit drudgereport.com, 2nd link from the top. I think this is the first time I've ever seen that!
Heh, looks like it took out a big portion of Bank of America's ATM (cash) machines! Link
This is not a dig at you, but since when did DMZ mean "completely unrestricted access to and from the Internet"?
I'm seeing this a lot lately, I think I first saw it when there were some of those home hardware NAT/router/dhcp boxes, if you put a server in "DMZ", it really means totally unprotected.
At work, I use a two firewall setup, and I call the area between the two firewalls the DMZ. The computers there have real routable IPs, but most of their ports are filtered, except the ones they need open. Every doc I read about firewall setup a few years back referred to these filtered hosts as DMZ also.
So when did the meaning change?
I've had enough abrasive sigs. Kittens are cute and fuzzy.
Yeah, it means rather than sending him to Cuba without a trial, the CIA will blow him up from a UAV without a trial
All's true that is mistrusted
Anyway, unnecessary: a database that has only open ports 22 and, say, 3306 (I like MySQL) is going to look very similar to a a machine behind a firewall that only lets port 22 and 3306 through. There are a few classes of DoS attack that could be stemmed through use of a firewall, but really, the value of your environment is your data. Run a sniffer on a compromised webserver, and you've almost certainly got the information you need to make backend connections to the database servers.
sloth jr
I think that the reason that a lot of these patches do not get applied is due to the "If it isn't broken, don't fix it" mentality. I know that many Microsoft Security patches in the past have caused say 1 out of 10 small volume custom applications to fail in some way after they were applied. The business being conducted by the application may have justified say a 50K dollar initial investment to have it written by a developer. However, the month-to month return does not justify paying a Maintenance fee in order to keep a developer up to speed on your code base. Microsoft has been releasing patches for either IIS, or SQL Server, or OS on roughly a schedule of 2-4 a month. Your average 10-50 man company that had an application written for their specific need is not going to be willing to pay you $4000.00 a month to maintain a secondary system with their application installed, 10-20 hours to test every single function, etc every time Microsoft releases a batch of patches. In their minds it's built, it works, and it's done and they are not going to pay a dime more. If you are lucky, they might do that when something like today's situation comes up. That is why most systems (I will even say Linux/Apache/XSQL systems) don't get every single patch that comes down the pipe applied. In a perfect world you would not accept the work unless there was a good maintenance fee included, but in the real world you take the work that people will give you and deal with the ongoing maintenance on a case-by-case basis. The only contracts where you get that kind of commitment is when there is EXTREMELY good revenue involved and the companies business absolutely relies on the application.
The current DDOS attack caused by a worm that exploits a known vulnerability (for which a patch was already available) raises the following questions :
a. Is this a test or preparatory exercise carried out before a serious of massive attacks due during the time US invades Iraq ?
b. Is there another vulnerability(ies) (probably bigger gaping holes) in the patch available for the current vulnerability which the group is hoping to exploit, during their second phase of attacks ?
These are just questions. I think administrators should be doubly sure about this patch before they apply it.
A Massive DDOS attack during the gulf war could cause:
a. Less or no information
b. With DNS servers down (5 down this time around) a massive disinformation campaign can be launched (Say the CNN site giving false information for a couple of hours)
These are just possibilities. So was September 11th.
My only question is that if this is so important, why do they banish it to parts unknown (pardon, the depths of their Technet site) rather than placing it in everybody's Start menu? Cheers to their security consciousness, jeers to their halfassed methods of information deployment.
Try not. Do or do not, there is no try.
-- Dr. Spock, stardate 2822-3.
Further evidence that MS is continuing to contribute to this problem:
15 out of 16 available versions of MS Desktop Engine, which is vulnerable to the attack, cannot be patched by any available download. You must purchase a CD-ROM and wait for it to be delivered.
From Section 2.2 of spreadme.htm from sql2kDesksp2.exe
When downloading and extracting the Desktop Engine SP2 installation file from the Internet, please use the following guidelines.
Download and extract the Desktop Engine SP2 file as described above for the Database Components and Analysis Services SP2 files, with the following exceptions.
If you download the Desktop Engine SP2 file from the Internet, you can apply the service pack only to instances of the Desktop Engine that were installed from sqlrun01.msi. If you attempt to apply the service pack to instances that were created using sqlrun02.msi â" sqlrun16.msi from the Setup.exe file that was downloaded from the Internet, you will receive one of the following errors:
This installation package could not be opened. Verify that the package exists and that you can access it, or contact the application vendor to verify that this is a valid Windows Installer package.
-or-
The upgrade patch cannot be installed by the Windows Installer service because the program to be upgraded may be missing, or the upgrade patch may update a different version of the program. Verify that the program to be upgraded exists on your computer and that you have the correct upgrade patch.
To upgrade instances of the Desktop Engine that were created using sqlrun02.msi - sqlrun16.msi, you must apply the service pack from the Microsoft CD-ROM. You can order the SQL Server 2000 SP2 CD-ROM from Microsoft by visiting the Microsoft SQL Server Downloads Web site.
One of the big problems with applying Microsoft patches, is that Microsoft uses patches to push unpopular and/or useless software on people.
For example, applying security hotfixes to Windows XP causes MSN Messenger to be installed, even if it was previously removed. This practice got a Microsoft infantry mobile-computing solution to be disqualified when Outlook Express and MSN Messenger were installed to Army XP-Embedded machines.
If you blindly apply MS patches to a mission-critical system, you're nuts. If you have the time to verify the multitude of MS patches as they come, you are probaly soon to be unemployed.
Conformity is the jailer of freedom and enemy of growth. -JFK
But if you ever have a change of heart, all you need to do is make a daemon that will respond to a 1434 UDP packet with an 04 in the first byte by sending a one-byte UDP 1434 response with an 08 as the data.