Slashdot Mirror
Fermi Lab Compromised by Pirate
tttonyyy writes "The US Department of Energy sounded a full scale alert after machines were compromised at the Fermi National Accelerator Laboratory, according to this BBC article. It turns out that the hacker was a student using the machines to download and store music and movies."
Not True. I work at IT another accelerator lab in the US, and the control network is on an entirely different network firewalled off, MAC restricted, etc. Even the software engineers responsible for the control system have to be wired behind the firewall.
On a not unrelated note, we have been hacked several times by people uploading movies, MP3s, etc. The system was never rebuilt and the files were simply deleted. In general accelerator labs are not staffed for the super-anal security that you would expect (to say nothing of the number of MP3s, etc. that legitimate users have on the server)...
I've worked at Fermi National Accelerator Lab (fnal.gov) for 4 years, so perhaps I could troll a bit: since they have so many Linux machines (nearly all on Internet accessable IP) and no firewall (recently there are some firewalled ports) this is not a unique occurance, this happens *all* the time.
On the other hand, FermiLab does no defense/weapon work or any kind or any classified work as far as I know, a lot of people confuse it with Argonne National Lab (and be really glad Argonne wasn't named an Accelerator Lab, otherwise we'd have anal.gov)
-frin
Here's what really happened. Users in one of the labs are all given web space on a web server. Now, the IT staff is low on manpower, with government funding behind diverted to the war in Iraq. So, security (among other things) is kind of lax.
Basically, McElroy ran Jack the Ripper on the password file. We're using an SGI 1400L from 1997. He got the root password, and removed the limits of his disk quota. Then, he stored a bunch of ripped DVD's and MP3's in his webspace.
Now you ask, why isn't the government making a big deal about this? They know their security policy is weak, and they just ramped it up. The 'alert' is really just a few days for them to get things back they way they should be. If they said "well, we won't prosecute him because if people really know what happened, it'd make us look bad", what would the American public (and rest of the world) think?!
heh, do you really think you can /. the bbc?
Have a look here to see their traffic. Totals are here. They can handle 2gb/sec. Thats some monster pipe, and it will take some severe slashdotting.
On the count of three, hit refresh like a mofo. If all 600,000 of us do it we might just create a tiny lump on that graph.
That's not to say that massive damage/downtime can't be done by breaking into the right machines.
This happened last year, he's only just been sentenced (by the british, not the americans). And this had nothing to do with the Patriot act. The reason he chose Fermi Labs is that he mistakenly thought it was a academic facility and so would not pay bandwidth fees (unis etc in England don't pay for bandwidth)
I'm not condoning his actions, just trying to clear up some of the FUD
Sorry but the Large Hadron Collider is being built at CERN in Europe. It is not at Fermilab, and even if it were the "controls" for it would not be on the same network as the experiments, each of which would have its own authentication hosts, etc. anyway.
There are thousands of computers at Fermilab, the vast majority which are desktop workstations running linux (logins are through Kerberos). Being your typical office computers sitting on a desk, they are connected to the internet via fairly high bandwidth. As we know, the WWW was invented in order for high-energy physicists to share data throughout the world, so not only does it not make sense for these machines to be cut off from the internet, it is an essential part of scientific research. Any machine that actually controls an aspect of an experiment (connected to any sort of particle accelerator or detector) is not likely to be connected to the internet.
So, yes, physicists and other scientists do depend on flawed technology, mostly because its the easiest way to be able to keep connected when you're dealing with large collaborations stretched across the world. The downside may be the occasional kid (wrongfully) taking advantage of a desktop machine attached to a T1 line. Where security is more vital, it is present. But its simply impossible to insure that everyone's desktop machine is secure or not.
Instead he ends up doing community service. Exeter is about half an hour from here. The community service in this part of the UK is an incredibly harsh and difficult punishment. I'll describe it for those who have not come across its horrors before.
Its likely that he will end up being forced to sit in a sunny field in the middle of the Devon countryside smoking joints and drinking cans of extra strong lager with all the other community service peeps, while they supposedly dig some ditch that doesn't need to be dug so nobody will ever care about it actually being done or not.
That'll learn 'im.
Fermilab actually does not do nuclear energy research(or at least they don't advertise it). They are mainly concerned with research surrounding their giant particle accelerator - like discovery and research of subatomic particles. I know because I've seen it.
Why does everybody seem to think that Fermilab is some kind of sensitive facility? News flash: Fermilab is a basic research facility, not a top secret weapons lab. Their security is lax because they really don't have anything to hide. All their results are available to the public anyway. After all, that is sort of the whole point of basic research. And it's not like the compromised computer was part of the control system or anything. Fermilab has a lot of computers. The place is huge.
Besides which, if you actually read about the case you'd realize that this guy had access to the computers anyway and all he did was crack the root password to increase his disk quota. Now, I'm not saying that's a good thing but it's more like abuse of a computer lab than anything.
Physics is good
It's funny that the article made so many claims about how firecely the DoE closed things down at the lab, and how they oversee nuclear weapons and such. Yes, Fermilab is funded by the DoE, and so they fall under the same rules for terrorist paranoia. But the lab has an extremely small amount of radioactive material on site. Mostly it's just small check sources and such for testing detectors. There are some slightly stronger sources for testing calorimeters, and I think there's even a tiny amount of Uranium, but not even close to a critical mass of the stuff. They do not have a nuclear reactor on site. No weapons research is being done there, only particle physics with the accellerator with a bit of astrophysics and neutron therapy on the side.
Fermilab has really been suffering from tighter restrictions since 9/11. They have a lot of community outreach programs, but these days it's not as easy for the public to visit the lab. They still can, but have to jump through a hoop or two. It's really too bad. It used to be completely open, and folks would often be seen fishing, hiking the prairie, or watching the geese and buffalo.
As for "confidential" material that a hacker could access -- The experiments are publicly funded and the data is all, technically, publicly available. But in practice the data from the various experiments is generally kept somewhat secure just so that physicists on competing experiments have a hard time "stealing" the data. But honestly, in order to make any sense at all of this kind of data you'll need a hundred people with an intimate knowledge of the experiment spending a few years on analysis. All experimental results are published in journals and are freely available on the web.
As for tampering with the data, well that's possible but to influence anything you'd have to be impossibly clever and hack systematic changes into tens of TB of raw data, which would require figuring out how it's packed, what it all means, and knowing enough physics to search for events and adjusting things accordingly. For any single individual, that's just impossible.
As for controlling the accellerator, that's ridiculous too. First of all, there is a lot of analog electronics controlling the thing in addition to the computers, and it requires a sizeable team of scientists to operate. I'm sure that someone could cause troubles if they wanted, but never any damage. The worst that could happen is for a magnet in the Tevatron to overheat and no longer superconduct. When that happens the magnet boils off a lot of liquid helium as a protective measure, and the beam wrecks into the sides of the beampipe. This happens every few weeks anyway, without the help of hackers. There is nothing dangerous you could do with the beam, aside from entering the tunnel and sticking your head near it. And to do that you have to cleverly defeat all the interlocks. I've even seen a curious colony of ants trip the interlocks, shutting down the accellerator for a couple hours.
The lab is big on security simply because it's funded by the DoE. There's no other good reason for it. It DOES tend to be a target for hackers because they have lots of computers, mostly running Linux (with pockets of AIX, IRIX, Solaris, OSF1, and VMS), and (forgive me) the computing staff really means well but often screws up. Nothing is a "decision", it's always a "policy". They have gone with Kerberos and some crypto-card thing which is entirely insecure. They must have known about the possible exploits of the crypto-card system years ago, as I knew someone who figured out a couple (very easily) and were trying to let the Fermilab computing group know about it in the best way. I think the blame lies in trying to keep telnet ports open for people who need to log in remotely from computers running Windows that do not have an SSH client installed.
> many of the machines at Fermilab are admin'ed by physics postdocs and grad students.
Yes but you forget to mention the rabid (to their credit) security team the lab has. The sniffers they have set up are effective.
It usually takes them less than 24 hours to identify a machine that has traffic patterns beyond the norm, often within one or two hours they can blackhole a port if warranted and hunt down the owner of the machine.
Previously I would have called it suicide to operate a largely unfirewalled network at a site of this scale, but they really do do a good job at it.
I'm actually a student at Exeter University and this IS old news. It was a big thing back in September in all the local and student newspapers. Plus the article is slightly misleading, as he wasn't a University student when the attack was committed, something the University is keen to point out.
First of all, it is not possible to log into any service at Fermilab without a Kerberos principal. ftp and telnet are not permitted, and there is an active security eam that scans ports on a continuous basis and will shut down any offending machine. There is no firewall because all traffic must be either outgoing web and data services or kerberized if incoming.
I have personally seen Windows machines shut down within minutes and their wireless cards confiscated when brought onto the site if a virus is detected. These scans are not optional to the user and are automatically performed. The fact that this user was caught and security tightened to prevent recurrences is proof that there is good security there. The comments above are almost all completely uneducated.
Finally, as noted above by some (few) intelligent readers, the story is old and is really about sentencing. there has been no recent compromise.
Troll-prevention note and disclaimer: For those who think the above or the story itself is an invitation to hack, I can point out that several such attempts occur per day, keeping the security team busy and alert, but that essentially all of them fail and the rare successful ones earn the attention of the FBI.
At least at Brookhaven NL, all the computers have a paragraph at login, "WARNING: The system you are using is property of the Department of Energy, it's not for use beyond your job, unautorized access == crucifixion, yadda yadda"...you'd have to be beyond retarded to not realize you're where you're not supposed to be. Machines on the internal network don't usually have .gov hostnames, though - just an IP, or the machine name.
Facts do not cease to exist because they are ignored. - Aldous Huxley
Why is a place like that connected to the Internet, anyway?
Umm, it's a high energy physics lab. It's full of high energy physicists doing research in collaboration with universities and labs around the world. It wouldn't be able to function without being accessible from people's home institutions.
Repeat after me:
1. There are no classified systems at Fermilab
2. There is no nuclear energy or nuclear weapons research at Fermilab
3. You can't control the accelerators from outside the lab. You can't even control the accelerators from outside the control rooms.
First off, thanks for writing this. I used to be the lead of the UNIX admin team at FERMILAB quite a few years ago. The people who've been writing all this drivel need to remember that FERMI has fewer "secrets" to hide from people than just about any small business has. Oh they have a lot of stuff that one team of scientists would just as soon the others didnt' see till they publish it, but nothing classified.
As to taking over the accelerator (lets just ignore the fact that the Hadron collider is on a different continent and not running yet) - You've obviously never watched a whole bunch of particle physicists spend a week trying to get something resembling a usable beam out of the accelerator. It's not like there's a "destroy the world" button on a web site. And trust me, that stuff isn't Internet accessible.
I also don't think that people understand that computer security at a site of this type is a continual battle over the security=1/convenience rule. The physicist-users want convenience. The computing staff wants security - it's pretty much like anywhere else. I was still having arguments with experimenters who didn't want to have passwords while I was there. I'm sure it's only somewhat better now.
You have to remember that FERMILAB is much more like a graduate school in many ways than it is a secure classified site.
The user directories have quotas -- either "hard set quotas" or "soft quotas" (simply messages from an astute sysadmin that you're taking up too damn much space and need to burn it off on CDs or something).
Employees can store large amounts of data (temporarily) in areas which are not backed up on the RAID servers (the ordinary user dir's are backed up), but it's wiped out every so often to keep it from filling up.
Most well-run labs I've encountered seem to work this way.