Greek Hackers Target CERN's LHC

Doomsayers Delight writes "The Telegraph reports that Greek hackers were able to gain momentary access to a CERN computer system of the Large Hadron Collider (LHC) while the first particles were zipping around the particle accelerator on September 10th. 'Scientists working at CERN, the organization that runs the vast smasher, were worried about what the hackers could do because they were "one step away" from the computer control system of one of the huge detectors of the machine, a vast magnet that weighs 12,500 tons, measuring around 21 meters in length and 15 meters wide/high. If they had hacked into a second computer network, they could have turned off parts of the vast detector and, said the insider, "it is hard enough to make these things work if no one is messing with it."'"

Comments

Why is that even possible?

[Reality Master 201]

Why can anyone get to the control systems for a piece of equipment like that from the internet?

Re:Why is that even possible?

[Anonymous Coward]

security updates?

Re:Why is that even possible?

[n dot l]

My understanding is they have the LHC linked to universities/research firms/supercomputers all over Europe simply in order to process the massive amount of data that thing generates. I might have read that wrong though. I've had nothing but trouble finding good information between the "BLACK HOLES, WE'RE ALL GONNA DIE!", the idiot reporters doing "human interest" style pieces about it, and the incomprehensible (to me) physics-babble.

Re:Why is that even possible?

[sakdoctor]

Vista needs to phone home for activation.

Re:Why is that even possible?

[Bogtha]

Yeah, right, as if the LHC has enough horsepower to run Vista!

Re:Why is that even possible?

[Anonymous Coward]

Hard to sneaker-net data to a > 1000 machine grid.

This system is a bit larger scale than you suspect.

Re:Why is that even possible?

[X0563511]

I know you are being funny, but CERN uses Scientific Linux.

SL is a Linux release put together by Fermilab, CERN, and various other labs and universities around the world. Its primary purpose is to reduce duplicated effort of the labs, and to have a common install base for the various experimenters.

Re:Why is that even possible?

[VJ42]

It's called "the grid". just do a google search for "LHC grid" and you'll get lots of info. Here's a couple of links for starers:
http://lcg.web.cern.ch/LCG/
http://www.gridpp.ac.uk/cernlcg.html

The BBC has a less tchnical piece on te grid: http://news.bbc.co.uk/1/hi/sci/tech/7534866.stm

Re:Why is that even possible?

[AlXtreme]

My understanding is they have the LHC linked to universities/research firms/supercomputers all over Europe simply in order to process the massive amount of data that thing generates.

You're correct (I did an internship recently on data management with the LCG/EGEE network). It's a massive multi-tiered network of datacenters (something like 50k nodes, 15PB of dedicated data storage, but don't quote me on these figures), all required to distribute the enormous amounts of data collected in the experiments to the researchers capable of processing the data.

I'm not going to be an ass and piss too much on the work of thousands of others, because it took quite a bit of effort to set this up, but them getting hacked doesn't really surprise me. The architecture they set up (even for only data-distribution) is very complex, and a lot of software they use has been written in-house or has been forked (years ago). Oh, and it's all open source, readily available for whoever looks for it. With the LHC being such a high-profile target, this is IMHO a security nightmare waiting to happen.

In what I've seen, I'm crossing my fingers that this break-in isn't related to the grid network, and that the next few months will go smoothly, but the grid has been primarily designed for high throughput, not security. Sure, they have certificates you need to access the grid systems, the policies are there, but technically I have my doubts.

Re:Why is that even possible?

[$RANDOMLUSER]

They expect to generate something like 27 Terabytes of data every day. All those ones and zeros weigh a ton!

No, the zeros don't weight anything.

Re:Why is that even possible?

[smolloy]

These things are internet connected in order to allow on-call technicians and facilities management to check/fix it remotely. All big machines work like this. Everyone claiming that it shouldn't be connected to the internet has never worked with a machine like this (PS: I have).

Seriously, we need to stop the hysteria over this. It's not like you're presented with a "destroy the world" button when you log in!

No, you'll land at a bash prompt. And then what? You won't know the commands necessary to get to the control system software, and, even if you did, you'd only be able to randomly tinker with magnets. This will either have no effect whatsoever, or will be prevented by the machine protection system.

The worst you can do is to interrupt operation for a while while they kick you out, and restore any changes you made (which would be easily done from automatic history software).

This is bad -- any crack like this is bad -- but it's not gonna cause black holes, it's not gonna release radioactivity, and it's not gonna break the machine.

People need to calm down.

Re:Why is that even possible?

[jd]

Open Source should have meant that a few thousand eyeballs scoured that code over the years between being written and being used. However, those few thousand eyeballs can't see code that nobody is told about. I discovered the source to the various projects by scouring CERN's network and digging deep through nests of links and obscure references. The Yahoo group for discussing grid computing has barely been used in the past year, and none of it for this. If there are any records for these projects on Freshmeat, it's because I added them. The project summaries are vague, where they're given at all. This simply isn't an acceptable way of distributing information. Their brief notices on minor pages away from the real information are about as useful as a house demolition order being posted in a basement with no stairs or lights.

Re:Why is that even possible?

[Oktober Sunset]

It looks like your creating a black hole...

Re:Why is that even possible?

[quanticle]

Unfortunately, it's not always that simple. Sure, there may be a few universities that have secure leased lines running to CERN, but, given the prominence of the LHC, there are sure to be researchers not from those universities who want to run LHC experiments. Cutting off those researchers simply because they're at the wrong institution doesn't seem fair.

Greek Hackers

[darth_MALL]

Any chance they had a Trojan Horse at the ready?

Well the world WOULDN'T have ended...

[KalvinB]

but some jackasses decided to mess with things they knew nothing about.

I'll get my towel.

Computer security rocket science

[Yetihehe]

See? See? Computer security is harder than building 27km ring with enough precision to smash single protons!

Can't we all just get along?

[mpoulton]

Can't geeks just be happy for society's scientific accomplishments and not try to screw up a good thing just because it's possible? Like the guy says, it's hard enough to make these things work when everyone's working together. Assholes.

LHC webcams

[GoNINzo]

I found an interesting video feed for the system they were accessing.

http://www.cyriak.co.uk/lhc/lhc-webcams.html

Watch it for a minute, you can see the effects the hackers are having on them.

Re:Air gap + Sneakernet

[SBacks]

Portable harddrives to move the data?

http://www.nsf.gov/discoveries/disc_summ.jsp?cntn_id=111420

This thing will generate 28000 TB of data per hour! Imagine the number of grad students it would take to transfer all those hard drives back and forth.

Re:The correct term

[zappepcs]

I'm with you on the nomenclature issue. Such an important experiment and mankind in general offers far too many whack jobs who want to shut it down.

The logic of the 'we're all gonna die' crowd eludes me. If nothing happens, all is good. If the world ends, doesn't matter anyway. All those that think they will go to meet their maker should be happy either way, right? WTF?

Did they use a Trojan?

[Gotung]

Just wondering if they used a trojan to gain access.

Sneaky Greeks.

Um, for all you jokers out there...

[rickb928]

Wondering why the LHC is connected to the Internet 'at all'...

Why was the Web even developed? Why was HTTP even thought of? Why was a graphical browser of any interest?

CERN. Ask Mr. Berners-Lee. And then contemplate the irony of wondering this at all.

Sadly, it looks like CERN needs to work on the security more, but hey, that's in the spirit of the World-Wide Wild Web, eh?

Air gap and 15 Petabytes of data annually

[fejes]

Ok, I know you want to think that this can be done... but how exactly do you air gap a system that produces 15 Petabytes of data annually and share that data with 100's of labs around the world?

By manual entry, copying this data across the air gap (120wpm) would take:

15,000,000,000,000,000 characters /(120 words/minute * 6 characters/word) = 4*10^7 years.

Even passing that back and forth on hard drives means shutting about (15Pb/365/24 = ) 1.7 Terabytes per hour. (24 hours a day.)

At some point, you have to admit that just connecting this thing to the internet and securing it is the right thing to do.

Re:what retard put it online?

[smolloy]

All these machines have connections to the internet. This allows on-call technicians to ssh in to debug a problem remotely, and for facilities management to make checks on the performance of the machine.

It's not like connecting to the control software will present you with a big red button labelled "Black Hole Generator". You'll be presented with a bash prompt, and, if you can figure out the right command, possibly a control screen that you don't understand.

These machines are stunningly complex, and the most likely outcome of some random script kiddie fucking with things is that *nothing* will happen. Someone more knowledgable (or lucky) might be able to find something that will be prevented by the machine protection system, or cause the machine to shut down for a while. Bad, but not as scary as you suggest.

Seriously. Anyone who thinks that random "hackers" can do any real damage, or that these machines shouldn't be on the internet, doesn't know anything about them.

(PS: I'm an accelerator physicist who has worked with several of these machines.)

Re:what retard put it online?

[JustinOpinion]

Are they anticipating researchers waking up at 1am and thinking "hey, I want to run one more experiment from home before I go to bed..."?

I know you were trying to joke... but the answer is probably "yes."

I've never worked at CERN, but it may be similar to large-scale science user facilities (e.g. x-ray synchrotrons) that I have worked at. Specifically, you want to be able to control the instruments remotely for a variety of reasons. Part of it is safety (in order to minimize time spent near radiation sources and industrial equipment). Part of it is convenience (to check on the status before driving all the way to the actual facility). Part of it is for collaboration (allowing an instrument scientist to log into the machine and change a setting for you, show you how to do something, etc.).

At many facilities, you can change samples, alter instrument settings, re-align, etc.; all without actually going to the facility. Scientists doing those kinds of experiments do indeed appreciate the ability to log into the machine at 1am and check on the status.

There are of course safeguards in place (e.g. hardware safety triggers that cannot be remotely over-ridden)... but it is sometimes possible to break something with remote commands. Now, most of the facilities that work like this are running samples, and need remote manipulation to switch samples and re-align and so forth. LHC doesn't have the same set of requirements... but there are indeed a variety of legitimate reasons why a scientist might need to remotely log into the system and change some settings.

Large facilities are designed to "do science" 24/7. Remote control is one thing that helps scientists maximize the usefulness of equipment. (Such as waking up at 1 am, checking on an experiment you started before leaving work, realizing the data is no good, fixing a few parameters, and running a new more useful experiment.)

you question the actions of the scientists?

[conspirator57]

remember: everything PhDs do is art. everything. including using their alma mater's mascot name as their password. art, i tell you!

Re:you question the actions of the scientists?

[Medievalist]

remember: everything PhDs do is art. everything. including using their alma mater's mascot name as their password. art, i tell you!

Years ago (when I still worked in science) I got a call from the US military. It seems one of our scientists was attacking one of their systems.

Since the scientist in question was on the other side of the world on a field trip at the time, it seemed likely that someone had compromised his account, and I shut it down.

When I eventually asked the scientist if was using a strong password, he was proud to recite a long dog-latin linnean binomial. It was very difficult to spell or pronounce.

Of course, that was also the first word you saw if you searched for his name on the Internet (using WAIS, since this was before commercial search engines). This particular scientist was the world's foremost authority on the organism with that difficult name, and had published dozens of papers on it.

To put it in modern geek terms, it was like this guy was Bill Gates, his userid was gates, and his password was microsoft.

The idea that criminal hackers might actually look up his name came as a total surprise to this world-famous scientist with multiple PhDs...

Misleading Telegraph

[hairykrishna]

The two key sentances of the article are:

"If they had hacked into a second computer network, they could have turned off parts of the vast detector "

"We have several levels of network, a general access network and a much tighter network for sensitive things that operate the LHC," said Gillies.

Basically they defaced a web page which is hosted on a server which is nothing to do with the LHC control network. Haven't we had enough ridiculous LHC scare stories yet?