Slashdot Mirror
Ongoing Linux/Solaris Compromise Epidemic
An anonymous reader writes to point out that Stanford's Information Technology Systems and Services "has written a summary of a series of compromises that have been happening at universities, research institutions, and high performance computing centers, for the last month or more. The attackers are using known vulnerabilities in Linux and Solaris, along with compromised user accounts, to gain access and control of systems, from standalone servers to HPC clusters ... (the attacks are still ongoing)."
going back to the back-door insertion attempt on the Kernel, the rooting of gnu.org's ftp server, the compromise of Debian's servers... it's the same people doing this.
Just a feeling.
I'm running a live cd distro based on Damn Small Linux. Is this the coming thing to prevent attacks and viruses from getting anywhere?
Nothing is written to a hard drive with this OS.
If so, how would this apply to the story on these attacks? How would anyone "gain control" of my computer under these circumstances.
BTW, Damn Small has a limit of 50 Mb, mine runs a little over 60 MB, and I put Mozilla Firefox and Wvdial in the remaster, as well as some office applications from the Debian list of over 8000 items.
The entire (up to date) Windows lab here gets compromised & backdoored to hell and everyone just says "Have it working by tommorrow". A Linux cluster gets compromised and they issue a press-conference.
my sig's at the bottom of the page.
I dont think we will ever have a fully secure box, these vulnerabilies will continue to pop up occassionally and there's nothing we (the developers) can do about that. It is just a testimony of the fact that we are imperfect beings and sooner or later we will have our errors exposed. It is not a bad thing, in the evolutionary way of dealing things, this (finding and sorting out bugs) could probably be a good thing. Having said that, I think developers do have control over how they respond to these problems, like coming up a problem that doesn't just band-aid the wound hoping to find a cure for in the future. Also developers have control over how fast they respond. On both criterias, open source peer reviewing is winner over closed sourced development. One tends to promote security through openness and and in the other security through obscurity like think MSFT( Read comments from a MSFT bigwig who said the only reason MSFT servers are compromised because the vulnerabilities are announced).
Activists United
Maybe they're doing it as a wake-up call for all. They seem to be busy and motivated, but still leaving enough evidence to alert many people as they go. Nothing malicous like deleting data has been reported, just a trail of root kits and exploits. It'll probably result is many more secure systems all round after it's over.
If more sysadmins installed this, perhaps we wouldn't have problems with so many Linux compromises? Of course it's no substitute for patching, but seems like a good additional security measure.
This is from the gnu.org software directory
The exploitation of buffer overflow and format string vulnerabilities in process stacks are a significant portion of security attacks. 'libsafe' is based on a middleware software layer that intercepts all function calls made to library functions known to be vulnerable. A substitute version of the corresponding function implements the original function in a way that ensures that any buffer overflows are contained within the current stack frame, which prevents attackers from overwriting the return address and hijacking the control flow of a running program.
The true benefit of using libsafe is protection against future attacks on programs not yet known to be vulnerable. The performance overhead of libsafe is negligible, it does not require changes to the OS, it works with existing binary programs, and it does not need access to the source code of defective programs, or recompilation or off-line processing of binaries.
Now, my opinion of MS is not that great, but this just seems wrong.
Regards,
John
Falling You - beautiful
As long as we are being consistent. If unpatched Windows boxes count when complaining about or keeping statistics on compromised systems then unpatched Linux boxes should count as well. Personally I believe Windows' perceived insecurity has more to do with poor administration than technical shortcomings, well at least with the NT family. Linux's intimidation of traditional PC users may work to Linux's benefit here, fewer PHB think they can have an "amateur" administer the Linux box as they believe they can do with the Windows box. If Linux becomes less intimidating we may find more "amateurs" administering them and find them about as vulnerable as the average Windows box. On the other hand, Mac OS X is an excellent example of what Linux could do if it ever gets over its "by geeks for geeks" attitude.
How does that differ from the worms which get released for Microsoft almost a year after the patch was released? I hear people railing Microsoft all the time for not 'getting it right the first time' when THAT happens...
Wrong. People rail because Microsoft rarely gets it right the first time, and are damned slow and arrogant about fixing security holes. Oh, sorry. They did speed up their response time on security issues after realizing that the public was noticing and they were losing a little market share in IIS.
The p;roblem, among others, is that we don't have enough real punishment going on for hacking activities.
The internet has become the equivalent to living in a slum. Sure, the property is cheap, but if you don't have bars on your windows, you can count on a break in. And lots of people will tell you it's your own fault for not putting bars on your windows and living in a walled compound with broken glass on the tops of the walls.
I agree that the systems should be patched, but the real problem is that there are communities of thugs who feel at liberty... NO, who ARE at liberty (due to the lack of a cohesive international enforcement) to do what ever they want to you machine.
I vote for real international difficult (I know that's not going to be trivial) and hard jail time when people are caught. And, just like Kevin Mitnick, they should not be allowed to work with computers when they get out.
Agile Artisans
Funny, the same argument is also heard when a new worm attacks an age-old-there's-a-patch-for-it Windows exploit.
Of course, most Windows users are clueless, so the Linux/Unix admins are pretty much guilty in this situation.
To confess (anonymously), where I work we are pretty slack about security as well.. we use ssh and pam, wasn't there a known security risk with these 2 a few months ago?
I can see why they would want to target academic boxen if they wanted high-powered computers to do some serious slaved number crunching. If they are just going to launch a DDoS attack or send a bunch of spam though, academic computers are not the best. Most academic sysadmins have fairly limited budgets, and spend a fair amount on bandwidth. As such, they rule their bandwidth with an iron fist in many cases. The Admins at my particular college have bandwidth flags on certain ports and a global flag of somewhere around 1gb/day over 3 days. Break that, and the admin gets very interested in what you are doing with your boxen.
I'm sure other colleges have similar schemes, and I've heard of many colleges which are even more strict with their bandwith (200mb/day limit, etc). These academic boxes may make good targets because of their relatively user intervention and user experience, but they don't have that great of a pipe on them, relatively speaking. If it was me, I would have gone after servers that also run wireless access points. Hard to tell where the bandwidth goes in some cases with those.
Even those who arrange and design shrubberies are under considerable economic stress at this period in history.
I still use libsafe. It is the greatest thing since sliced bread. Ok, that and distcc. Distcc and rsync... and ssh... DOH!
"Learning is not compulsory... neither is survival."
--Dr.W.Edwards Deming
He didn't say 'require unmemorizeable passwords', just 'require passwords with characteristics that make them difficult to crack'.
An excellent point, however, that that standard 1337 letter-number substitutions do basically nothing to improve your password security, as any half-decent password cracker will try those substitutions early in a dictionary attack.
I recommend the use of symbols where appropriate (throwing a !, ^ or & into your password won't hurt) and taking the time to try to pick a good password -that you can remember-. Playing the 'license plate game' or using phrases or mnemonics can be a good way to generate memorable yet difficult passwords.
Example 1: "h8red&NV" (hatred and envy)
Example 2: "9.8m/s/s" (g)
Example 3: "wm$ihaBp" (with more money, I'd have a better password)
Example 4: "qP*&^%Zm" (letters from the four corners of a qwerty keyboard, with shifted '8765' in the middle... try it, it types surprisingly easily)
WARNING: DO NOT USE ANY OF THESE EXAMPLES AS IS.
so do you think that its illegal to pick something up off of the sidewalk ?
First and foremost "hacking" activities as you so aptly put it, are not the reason this is a problem, its the LACK of hacking activities at companies like MS that started this problem, they dont check their own software well enough. period.
A hacker doesnt break the law (well any sane law, shit like the DMCA can fuck off) script kiddies and crackers are the one's who do shit like this.
If you leave your system wide open its like owning a retail space, and not having a clerk, or prices on anything. People will (rightly or wrongly) assume the merchandise is free if there are no prices, or methods of checkout. Leaving a system (any) wide open like that is where you get into trouble, its not B&E if there is no B.
The laws are already in place, have been for years and they are tough enough (5 years for causing damage is plenty, unless you think your average teen deserves life ?) very rarely do these problems result in real damage, mostly its "possible" or "potential" damage, much the same way spilling a milkshake on those gap jeans at the mall is, its easily repairable, and the responsibility lies on the store keep for allowing the shake in the store, and the person who spilled it.
"Two things are infinite: the universe and human stupidity; and I'm not sure about the the universe." --Albert Einstein
I see a day coming when, in one day, half the computers in the US have their disks erased.
its the LACK of hacking activities at companies like MS that started this problem
According to a friend who used to work there, MS has teams of people whose job is to take their custom-built equipment anywhere they want on site and see if they can hack into systems.
I'm not really sure what more they could be doing, other than allowing everybody to view their source code.
"...always new atoms but always doing the same dance, remembering what the dance was yesterday." -Richard Feynman
Any admin needs to abolish telnet and unencrypted http passwords. The first step in security is the password layer and they're just wasting that. The privlidge elevation bugs are a problem, but they should be keeping important servers up to date with only a short testing delay. Linux may be secure, but it's by no means perfect.
I like the glass analogy for pointing out that the hacker is still the one at fault. But I still think it valid to say the admins who weren't patching are still at some fault. At the very least it's more reasonable to blame them than the OS developers if the fix had been available.
To follow you analogy, blaming the developer for a break in via an old, known & fixed bug would be like blaiming the fellow who installed the window months ago because so thug put his fist through it.
Better analogy might be blaming Ford because your wheel fell off your car months after they sent you a recall notice for the problem. They made the initial mistake, but your at fault if they tell you and offer a fix that you ignore.
Granted, Microsoft takes a lot more heat than most vendors in these cases, but I think a healthy amount of that can be chalked up to social karma. They're big and a lot of people believe they did dirty things to get there. It takes decades and honest effort to live that sort of thing down. Also it appears to many that Microsoft has a greater number of severe vunerablities, that they have a history of treating it lightly and that it is too often a design flaw at the root of the problems.
Well... yeah, they are, what's really the problem with admitting that? We know something about the company and their track-record, why should that not be allowed to colour our current opinion of them?
sic transit gloria mundi
Yeah, I've been involved in some of the staff discussions at one of the compromised institutions. The vulnerabilities listed seem old because these attacks have been ongoing for a while now. Some of those vulnerabilities were actually discovered originally in relations to this situation. What's important to realize is that this situation is very unlike what's happened to windows machines recently. Most of the Windows intrusions have been remote exploits via services. We've been facing primarily local-root exploits. These people are breaking into accounts--usually by password sniffing, key-stroke logging, etc from other compromised machines. Those accounts are then used to launch various known (and previously unknown) local-root exploits. These people appear to be after other systems for an unknown purpose rather than just "games" or DoS attacks. Most of the targeted institutions have substanial DARPA/government research contracts. It's reasonable that these attacks are being used to steal information. The focus has not been on High Performance Clusters but rather on interactive clusters. These people are after information not computing power.
I've heard a lot of people say something like, "It's their own fault for not installing the latest patches." Doesn't that suck anyway though? It's a major pain to need to keep a human around to twiddle some bits periodically.
I'm not sure it really has to be this way. It seems to me, that it is a major design flaw that if there is a small error in one of the *many* programs from *many* different parties being run as root, that it can be exploited so that an arbitrary attacker can end up getting root access or executing arbitrary code or whatever. For that matter, it seems silly that (for desktop systems) disastrous effects can come from code run by Joe user. After all, desktop users store all their important files in some place they *don't* have to authenticate as root to get to.
Rather than just assuming that the ever watchful eyes of open source uber hackers are the only remedy for this as well as all of life's problems, maybe it is possible to come up with some easy solutions, or at least partial solutions, to this problem?
1. Use software that watches the beginning and end of every stack frame for an overflow. If an app overflows *kill it dead*. Similarly, the beginning and end of every block allocated on the heap can be watched. Software like this exists, and it is about time it is built directly into the standard distributions and *turned on by default*.
2. Develop a new security model. The current system sucks out loud. Really, access lists (a la microsoft) are a step in the right direction. Finer grained and more flexible controls are good, but a totally new security model would be better. I've seen some things like this developed as academic projects, but it would be nice to see a patch available for a main stream OS like linux.
3. It might also be useful to have virtualization (think VMWare) built into standard distros and used by default for services like apache that need to run some stuff as root. My understanding is that you can do something like this with chroot currently, but that it is a clumsy and dangerous tool.
I'm not a big security buff, but even I can see that there are some things we can actually *do* about this problem.
The thing to remember with cd-based distros is that, even though the media cannot be changed, many things that are stored in writable memory can be, up to and including the system BIOS. It's a good idea to reboot them periodically to verify that you're working with a "clean" OS and that any intrusions or modifications have been reverted.
"I assumed blithely that there were no elves out there in the darkness"
I can confirm this is for real. NCSA, SDSC, ANL, Caltech, and other sites have been hacked, largely Teragrid and HPC resources. .mil has even blocked access from some university nets to prevent attacks, because these attackers are targetting universities for their high-performance resources (for password cracking) and plethora of DOE/DOD and researcher accounts, some of whom have access to classified systems such as Frost at LLNL and the ASCI systems at LANL, Sandia, etc.
They (especially the Teragrid folks) have been trying to keep it very quiet. However, if you read between the lines of this memo:
http://www.teragrid.org/userinfo/index.html
Basically, they've got several clusters at NCSA offline, and accounts being used by crackers to gain access to other systems.
This is the biggest university hack/government hack since the UofO & DOE/LANL hack last winter. Oh, yeah, you never heard about that one either, did you... I mean, if you knew how broken all these sites are, you might be shocked.
More info on the .mil block:
http://www.its.caltech.edu/
So if I put -fstack-protector in my global CFLAGS, I can ignore all the critical buffer overflow exploit warnings? Why isn't it on by default?
I mod down anyone who says "I will be modded down for this", regardless of the rest of their comment
we have a sun system at our institution that runs a webserver for a very specific application. an unnamed vendor (we'll keep it that way) installed this machine and pretty much told us to keep hands-off of it except to change the backup tape. if we made any modifications to the machine or its software, then our service agreement was void and they would not support this particular app. so, we firewall the crap out of this thing, only allowing access to httpd (apache), making sure to explictly block any high port in use. well, this machine gets compromised about a week ago because this vendor has an ancient version of apache (1.3.3 or something) running suid/sgid root. idiots.......this is a problem we could have prevented if our vendor wasnt as dumb as they were. being a small .edu, we cant just pack up and change without spending 6 figures, so we are pretty much stuck with it until their contract comes up in a couple of years (this is an inherited problem). want their take on the problem - apache only will work suid/sgid. wont run unless permissions are that way. so i ask them to change it, and after about 10 minutes of arguing with their lead UNIX guy he does so. he was amazed that it would run......