NASA Nebula, Cloud Computing In a Container

1sockchuck writes "NASA has built its Nebula cloud computing platform inside a data center container so it can add capacity quickly, bringing extra containers online in 120 days. Nebula will provide on-demand computing power for NASA researchers managing large data sets and image repositories. 'Nebula has been designed to automatically increase the computing power and storage available to science- and data-oriented web applications as demand rises,' explains NASA's Chris Kemp. NASA has created the project using open source components and will release Nebula back to the open source community. 'Hopefully we can provide a good example of a successful large-scale open source project in the government and pave the way for similar projects in other agencies,' the Nebula team writes on its blog."

Comments

Of course they will

[NoYob]

In addition, the Nebula platform itself will help facilitate the adoption of open source software across the Government.

That won't be sole the reason. As departments have to cut budget's in the near future, they'll be looking more and more to F/OSS to save money. Nebula is proving the low budget F/OSS solution as viable.

Re:Of course they will

[Mortaegus]

It doesn't matter what the reason is, the fact that they are doing it at all is what matters to me. I'd accept a company bragging and boasting about themselves for doing something good in a heartbeat, as long as they did something worth doing. Likewise, I don't care what the motivating factor is in this instance, as long as it gets done, and really is open-sourced.

quick?

[pak9rabid]

NASA has built its Nebula cloud computing platform inside a data center container so it can add capacity quickly, bringing extra containers online in 120 days

4 months is quick?

wrong use of "cloud?"

[Lord Ender]

Like so many promising high-tech ideas, the word "cloud computing" is being over-used. Cloud computing means being able to get virtual hosting with a few clicks, and automatically scale up and down as demand changes, all while being billed by resources actually used.

Not every cluster of servers or supercomputer deserves to be called "cloud." Not everyone who runs VMware deserves to be called "cloud."

NASA computing in a cloud?

[Subm]

A nasa computer in a nebulous cloud?

I think we all know where this is going -- V'ger wants to bond with its creator and instead gets a bald girl and Decker to create a new life form.

Is NASA suffering from mission creep?

[wisebabo]

Look, I'm a big fan of the space program and everything but with the country at (two) war(s), hocked to the hilt, economically stuttering, NASA (like the rest of the government) needs to be focused on its "core competencies" (no I'm not a PHB). Where does building data centers fit into NASA's mission statement?

I realize that there are tremendous amounts of data that needs to be captured, analyzed and archived (the Terra satellite sends a terabyte of data a day alone I think) but isn't this something that can be done more efficiently by private industry (Google?). Maybe it can be even outsourced providing it is not of a sensitive nature, I mean isn't the data for all mankind?

Re:Is NASA suffering from mission creep?

[belthize]

A lot of their information is considered sensitive, particularly engine modeling and telemetry.

The question of mission creep is certainly valid. In this case though I suspect it's simplest/best for NASA to do it themselves. Frequently outsourced cloud computing at this level looks good financially but simply isn't practical when you translate white paper speak:

extensible, flexible, segmented, secure, etc into reality

use our API and like it, you can have 8 or 16GB mem/node, somebody else is using the cluster you get these 0 to N nodes now, sorry about your data getting out

The right approach here (and one that NASA appears to be taking) is sharing of knowledge on how best to build a flexible cluster environment. It's OK for everyone to have a wheel, we just need to quit deciding if it should be round or not.

I have to say I cringed reading some of the Nebula pages. They're definitely written by somebody trying to sell the project to the rest of NASA. That's the real danger, not that NASA develops it's own cloud but that NASA is so departmentalized that nobody uses it and each sub-section develops their own.

Re:Is NASA suffering from mission creep?

[DerekLyons]

NASA (like the rest of the government) needs to be focused on its "core competencies" (no I'm not a PHB).

If you aren't a PHB, then why complain using PHB/Marketdroid fuzzy buzzwords?
 

Where does building data centers fit into NASA's mission statement?

NASA does all manner of things that aren't launching things into space because without doing those things, the things they do launch into space might as well be chunks of firewood. For example, operating a large communications network, or operating a considerable chunk of data processing horsepower. But, you need to read TFA - they didn't develop a data center, they took commercially available hardware and deployed the [open source!] NEBULA cloud managment application on that hardware.
 

I realize that there are tremendous amounts of data that needs to be captured, analyzed and archived (the Terra satellite sends a terabyte of data a day alone I think) but isn't this something that can be done more efficiently by private industry (Google?). Maybe it can be even outsourced providing it is not of a sensitive nature, I mean isn't the data for all mankind?

Actually, given the byzantine nature of Federal procurement, it may not be more efficient to be done by an outside contractor. Doubly so given the even more byzantine web of privacy, access, and security requirements.
 
Yes, the science data is for all mankind, but there is usually a 'hold back period' of a year or two where only the science team (usually from outside NASA) has access to it. This is only fair, as they're the guys who fought for funding for the instrument, designed it, developed it, tested it, operated it - and spent years of their lives doing so. (We're used to talking about 'NASA satellites', but in reality NASA is often just the bus driver and road crew for instruments from outside of NASA.)

Re:Is NASA suffering from mission creep?

[Angst Badger]

but isn't this something that can be done more efficiently by private industry

I know that there is, in some areas, a belief in the unicorns-and-rainbows magical power of private industry to perform more efficiently -- a view that makes one wonder how many of its adherents have actually held real jobs in private industry -- but the evidence is simply lacking, particularly where government contracts are concerned. Ever seen the miracles private industry works with DoD contracts?

The vital difference between the public and private sectors that market ideologues always fail to take into account is that private businesses have to turn a profit. That's an expense that not-for-profit activities don't incur. Given the choice between scientists who care primarily about the outcome of the project and private contractors who care primarily about billing for more than the cost of the project so they can make a profit, it's far from clear that the latter will always do the best job.

Re:Is NASA suffering from mission creep?

[AMuse]

To follow up on this (Disclaimer: I am a NASA employee), NASA and other federal agencies are prohibited by policy and law from transmitting or storing many of our data types on non-government owned hardware and networks. (Transmitting of course can be done if it's tightly encrypted). Processing our data on private servers is strictly prohibited in many cases.

The most frequently cited laws and policies which dictate this are FISMA and OMB M-06-16, but there are many others. Employees are even prohibited from doing team collaboration with things like Google Docs, because information which is not yet deemed to be sensitive (say, an immature design for a propulsion system) might become very sensitive, and once it's "out" it is out for good.

Like it or not, there's a lot of other countries with developing missile programs, communications programs and many other technologies which have dual civilian and military use, and NASA is charged by congress with keeping technology that may have military applications out of foreign hands.

If Nebula is able to perform as well as clouds such as EC2 and the like, and allow NASA and other federal agencies to do cloud style processing within the government sector, it could save HUGE amounts of taxpayer money that's otherwise legally obligated to be "Wasted".

Re:Is NASA suffering from mission creep?

[AMuse]

There's another important factor in the paranoia about data breaches and risk that's often VERY overlooked.

As part of the chain of responsibility, the CIO community (the individual CIOs at the 11 NASA centers, and the federal CIOs in general) are very risk-averse. Why might that be? Well, in addition to the normal slamming your agency has to endure if there's a data/privacy breach, the CIOs and decision makers may also be civilly or criminally liable for negligence if it can be shown that they were permitting workplace practices that went against federal regulations. A few CIOs that I know are actually carrying personal liability insurance (out of their own pockets) to cover themselves in case such accusations are leveled.

Now, imagine you're the person tasked with pushing the envelope technologically (Hey, it's what NASA does) but the only thing your bosses ever remind you of is that it's your ass on the line if anything is ever breached, inappropriately stored or transmitted, etc -- and that fines and jail time aren't out of the question. That's enough to make someone pretty risk-averse!

Can You Imagine a Beowulf Cluster of These?

[vrmlguy]

Hello, old meme, long time no see.

They're using Eucalyptus

[Xouba]

The best thing about this is that they are, allegedly, using Eucalyptus: http://nebula.nasa.gov/blog/2009/nov/how-eucalyptus-enables-ec2-compatibility-with-nebu/

Re:I don't get it.

[drinkypoo]

Shipping containers are stackable. Remember when blade servers were introduced, and many people said "no one will want those, you can't even install per-node peripherals"? These are the blade server equivalent of data centers. Also, all ISO shipping containers have convenient mounting points, so you can protect these from heavy weather by sinking piles and just bolting them down. They even have the potential to be watertight, so if you built them cleverly enough, with air intakes and exhausts bussed through them and with hoods on the roof of the top unit, they could even be flood-resistant in such a scenario.

I have no idea if any of that applies to this system, but there's good reasons to use containers.