Slashdot Mirror
What Kind of Cloud Computing Project Costs $32M?
coondoggie writes "The US Department of Energy said today it will spend $32 million on a project that will deploy a large cloud computing test bed with thousands of Intel Nehalem CPU cores and explore commercial offerings from Amazon, Microsoft and Google. Ultimately, the project, known as Magellan, will look at cloud computing as a cost-effective and energy-efficient way for scientists to accelerate discoveries in a variety of disciplines, including analysis of scientific data sets in biology, climate change and physics, the DOE stated. Magellan will explore whether cloud computing can help meet the overwhelming demand for scientific computing. Although computation is an increasingly important tool for scientific discovery, and DOE operates some of the world's most powerful supercomputers, not all research applications require such massive computing power. The number of scientists who would benefit from mid-range computing far exceeds the amount of available resources, the DEO stated."
. . . but also the rest of the sky including the moon and the stars.
One CPU cycle wasted on digital restrictions management is ONE TOO MANY.
This is the government.
What kind of [random project variable here] project costs less than $32m?
"The number of scientists who would benefit from mid-range computing far exceeds the amount of available resources, the DEO stated."
This sounds like one of those far-fetched statements that more realistically would be answered as "eleventy-billion."
The question is not "What kind of cloud computing project costs $32M?" The question is "Is research into the benefits of cloud computing worth $32M?"
As with many multi-million research grants, it looks less like valuable research and more like a handout.
You know, usually I'm against most government spending programs. They tend to be a huge waste.
But this... It sounds interesting and could actually benefit basic research- something this country sorely needs to support. My (perhaps incorrect) observation is that some groups like the DOE and DARPA tend to allocate funds to valuable research projects rather than pissing money away on terrible administrative database implementations. I guess I should keep in mind that the majority of DOE funding is used to build and maintain our nuclear weapons fleet.
Clearly they're trying to create the Large Magellanic Cloud.
...would that be mushroom cloud computing?
Yeah, when I saw the headline I was prepared for a trainwreck, but this actually sounds like a really good government project. The ultimate Beowulf cluster, at the disposal of all scientists in the nation.
With that much money they could get a quarter of an F-22 fighter jet! How dare they spend it on research?
The DOE stated.
The kind where the company who receives the contract is located in a particular Representative's district.
"As God is my witness, I thought turkeys could fly." A. Carlson
I imagine a large portion of that cost are salaries.
The trouble with supercomputing is that, if you have to share the thing, you don't need it.
Supercomputers are worth the trouble if there are applications that need hours or days of time. But if you have many users sharing the thing, it's a waste. Price/performance tends to be maximized towards the upper end of mainstream machines. Supercomputers, with their custom hardware, tend to have lower price/performance than commodity machines. That's why web farms are made of commodity hardware.
... the DOE stated.
Just call it the Large Magellanic Cloud
http://alternatives.rzero.com/
What's the bright side of cloud computing?
When the cloud goes down, it's a bright and sunny day.
LOAD "SIG",8,1
LOADING...
READY.
RUN
... sounds like a walk in the park compared to their other spending. I think that number is off by a factor of 100 or so.
In contrast, my small city (~40,000 people) in central Canada is spending ~$56,000,000 on a new Multiplex/Sports center. Supposed to have a new hockey rink, curling rinks, soccer area's with artificial turf.
I'd my city council spend it on a Cloud Computing Centre.
Not really. Pretty much all the big cloud computing companies build on open source, not just because it is cheaper, but because it is also better suited for the task / more adaptable.
The application software for big science related calculations isn't exactly off the shelf either, most of it is custom made.
Once you put together this kind of project, you can also hire some developers to build a software that runs on it, and are no longer restricted by home / small business development / deployment barriers.
Beowulf cluster of....
Oh, wait...
Never mind.
--- Asking inconvenient questions for over 30 years...
Plus the thousands of other reasonable-sounding government funded projects that cost less than a dollar per taxpayer...
Can I have $.21 from you? Just once, I promise. And no-one else is going to want the same, I assure you.
This is the same reasoning that allows $x.99 to be such a successful marketing ploy. Have you ever heard the phrase nickel and dimed to death?
Oh, was that my outside voice?
The specifications for that cloud include a silver lining.
You cannot truly appreciate Dilbert until you read it in the original Klingon.
is smoke, and the project was titled "Burning 32 millons"
SETI works on what gets described in the trade as "embarrassingly parallel" problems - supercomputers deal with stuff that's harder to get good parallel speedup without throwing fancy hardware at it. DOE problems are often somewhere in between, and unfortunately the boinc/seti/screensaver approach to ad-hoc supercomputing isn't always good for applications like LINPAK, so it's hard to compare the real computing power. However, if you ignore that (:-), most of the top computers in the Top500 list are doing nuclear or military work, and some are for weather bureaus or Earth simulation, but about half of the last decade, the SETI people are volunteering 2-10x as much CPU just to look for little green men as the largest military supercomputers were providing. The supercomputer guys are back on top, and I don't know that we'll catch up with them again, but on the other hand some of them are now doing genetics and other Good Things instead of Evil.
Bill Stewart
New Fast-Compression-only CPR http://preview.tinyurl.com/dy575ks
When the last ATC project failed disastrously, people were already playing online games over phone modems. Now we have massively multiplayer games, with gigahertz hardware dedicated to each user (your PC, that is), and ATC is still being done on single mainframes. Quick scan suggests six thousand planes in the air at a time over the US; let's call it ten thousand. Dedicate a CPU to each plus some hierarchy of busy areas and regional control; allow $1000 per CPU/system (and its share of comm bandwidth); call it $10 million. Sounds like an interesting project. :-)
There are problems which really need high memory bandwidth and don't fit on smaller-than-super computers, so a time slice on a supercomputer can be worth far more than full-time access to dedicated fast conventional computers. But those problems become less and less common as regular computers get bigger and faster - your laptop probably has a graphics processor that's faster than a Cray-2 by now...
Bill Stewart
New Fast-Compression-only CPR http://preview.tinyurl.com/dy575ks
I was wondering if I was the only one who noticed the overuse of that statement.
Apparently wizard is not a legitimate career path, so I chose programmer instead.
I will gladly give you $0.21 if I (and the many generations after me) get something useful in return. Like the Internet infrastructure we are all using right now.
I judt got a nre Kinesis keybiartf so please excusr ant egregiou typos.
and remember, kids: this thread was brought to you by a 40-year-old DARPA project.
We already have a platform to do this - BOINC. We've been wasting megawatts on SETI for years. Perhaps we should turn the search closer to home and just search for terrestrial intelligence, but that could be equally futile.
Easy: the one where you are building the cloud.
Makes sense to me!
You're not old until regret takes the place of your dreams.
The difference is those projects usually cost less than a dollar year after year, forever. This one would cost less than a dollar once. Then it would be funded privately.
I judt got a nre Kinesis keybiartf so please excusr ant egregiou typos.
As the saying goes: keep your feet on the ground instead of your head in the clouds.
Sounds like the old John Earling ("Earling in the Morning") routine on KRMG. He was making fun of Oral Roberts, but it was, "Send me your dimes. If everyone within the sound of my voice would send me a single dime, I would be a millionaire, and you would be only a dime short."
Why, without your clothes, you're naked, Miss Dudley!
BOINC doesn't work so well if the tasks need to download 10-20 GB of data and the actual applications running the job take up another 10GB of space and the jobs run full out on a system for 2-3 days while consuming 2GB of memory.
"When you sit with a nice girl for two hours, it seems like two minutes. When you sit on a hot stove for two minutes, it
----------------------------------- My Other Sig Is Hilarious -----------------------------------
In socialist America, children go to school and learn something useful, everyone has healthcare, the entire planet doesn't see the US as a meddling bully that resorts to violence to solve all of its problems, and technology is seen as an opportunity rather than a nuisance. Oh, the horror!
weinersmith
The difference is those projects usually cost less than a dollar year after year, forever. This one would cost less than a dollar once. Then it would be funded privately.
I'd imagine it'll take 3 to 5 centuries to match 6 months of just of one's personal vice habits, ala Starbucks, Cigs, Booze, etc.
I (and the many generations after me) get something useful in return.
That's the real caveat, isn't it? Things like Social Security were great for a few generations, but before long you'll have to be above the average lifespan to collect because it is going broke. Never mind the fact that that single program alone accounts for about 1/3 of the US deficit. Think about that for a minute - you have to be 65 to collect, and the average life span is in the upper 70's. It's 1/3 of our national debt, yet it will only cover a little more than 1/10th the average citizen's lifetime. It's benefiting the current generation at the expense of the next, and it's exactly the sort of thing people are afraid of with any large government spending project.
The real insidious thing is the hundreds, if not thousands of $32 million projects that fail, and we end up paying for with nothing to show for it. They each individually are too small to take much notice (even $32 has me going "meh" as far as size of project to be worried about), but taken together they represent massive waste.
As far as this particular project, the hardware costs are probably not more than $1 million, it will probably cost $5-10 million to design the system, which is justifiable, and then the other $21 million are all administrative costs. Then the project will over-run when the people running the project change their minds halfway through (and then again change their minds back, or just to something completely different), causing the engineering costs to skyrocket, which in turn causes the administrative costs to skyrocket. I wouldn't be all that surprised if this $32 million project ends up costing $70 million. It happens all the time.
Security is mostly a superstition... Avoiding danger is no safer in the long run than outright exposure. - Helen Keller
...is upon the clouds! Sounds more like distributive computing rather than online storage/file hosting which is really all cloud computing will amount to
I came a little after reading this.
Visit my Forums?
And the final misspelling DEO.
Don't blame me for redundant posts. I can't type very fast. Hence the user ID.
I agree that the SSA is a system built for 1930's America. But I guess the principle you state of "It's benefiting the current generation at the expense of the next..." I disagree with. I don't know about you but for the first nearly two decades of my life I was wholly dependent on the then current generation to feed, house, and clothe me. And then for the next decade or so after that, they continued to provide me with employment, and subsidize my car insurance and higher education. I guess I feel I feel it's more like repaying the current generation at the expense of the next.
I judt got a nre Kinesis keybiartf so please excusr ant egregiou typos.
you're welcome.
send the check to 1 Lenin rd., Stalingrad, RU 82317.
weinersmith
Here's a DOE Lab site. Fargin' Iceholes!
I drank what? -- Socrates
It is NOT going broke. That's a myth thats been perpetrated since it's inception.
I remember when I was a teen, it was supposed to ahve been completly over whelmed by 85, then 2000, now it's 2015.
Read up nio the works of the peple that actually study it. It need MINOR adjustment from time to time but it
s not going to collapse.
Well over 99% of all federal project succeed, on time and within budget, and with less waste.
Failed projects do not equal waste.
"the hardware costs are probably not more than $1 million,"
for a project this size? you clearly have no experience building out systems.
We are tlaking about thousands of systems, and good ones not POS bottom of the line Dell's.
You need to pay for the infrastructure. Back bone, racks buildings and other sunk costs.
(Are you lumping this into administrative?)
Now we need people. They are using linux, so probably 1 fte per 200 machines.
Then system design.
Quite frankly, this is a good price for what they nede to do.
Maybe there will be 'cost over runs'. Over runs are often do to provider cost changes. Contract where something is delivered years after the beginning often have a clause to allow more money to cover those costs. I am talking about hard costs, cabling, concrete, etc . . .
The bidest example is rock. The price of rock can be volatile, so it's not uncommon to see bids where they amount paid in the contract is adjusted to cover the providers cost. If you don't do this, bids would be nearly impossible.
"It happens all the time."
no, but the bias is that it does because the 10,000 times it doesn't happen no one says anything.
I was in the private sector for a great many years, in the few years I've been in the public sector o have been constantly amazed at the tight book keeping, the amount of knowledge people have, the accountability, the incredibly high skill set.
Turns out there are very smart, dedicated and qualified people who take a government job becasue they are tired of not having a life.
The Kruger Dunning explains most post on
Actually if you understood why SSA is going broke it isn't because of a funding it is because politicaians can't let money sit for time until it's needed.
Basically every up until recently the SSA had been making massive amounts of profits compared to what is paid out. However at the end of the year instead of putting that money safely into a bank account for the next generation to actually use the USA government claimed the money and wrote an IOU to the SSA for said money. our politicians then spent said money on random projects. However we are approaching crunch time for when the money that was supposed to be saved for our future, is going to be needed and the SSA has realized the federal IOU's were written on political toilet paper.
As usual it is careless thinking on politicians that have caused this mess. For whatever reason politicians are the kinds of people who think maxing out your credit cards, and loans is a good way to live.
i thought once I was found, but it was only a dream.
One that includes Microsoft
Who would win this election: Andrew Weiner vs Andrew Weiner's weiner.
What kind? The kind that requires a building that sits on land and is full of hardware.
$32 million isn't that much when you consider that.
Here is an estimate for an empty 80000 square foot office building with no contents and no land. ~$12 million.
http://www.reedconstructiondata.com/rsmeans/models/offices3/
"Never mind the fact that that single program alone accounts for about 1/3 of the US deficit."
Not true. in 2010, SS will add $10B to the deficit. The deficit is projected to be above $1T, resulting in SS being 1%, not 33% as you claim, of the US deficit.
http://www.google.com/hostednews/ap/article/ALeqM5h6BfoloJOnV0TeI7eIHC1ZWuBxygD9AVLTVO0
"Think about that for a minute - you have to be 65 to collect, and the average life span is in the upper 70's"
Average life span is a tricky measure. Many people die as the very young or as teenagers. A much smaller percentage of people die between 18 and 80. If you've started work, there's a better than even chance of you collecting your social security.
Citizens Against Plate Tectonics
Most people see the 32 million dollar price tag and have this knee jerk reaction. They don't take into consideration the people they will have to pay, hardware, facilities, and all that other junk that goes into a large project. I stopped getting all worked up when I learned that a project I worked on cost around 200 million.
This is the best summary of project costs I've read. It applies, as the author said, to private as well as public projects.
Citizens Against Plate Tectonics
No, socialism does not make the military industrial complex weaker.
32 mil !!! Thats nuts
No, what they really want to do is build Skynet.
You all still think the earth is less than 10,000 years old though, right?
i like the way you think:
not resorting to violence => inability to do so => weakness
i suppose this is a logical conclusion from the following way of thinking:
strength => ability to blow to smithereens anyone, anywhere => desire to do so incessantly
weinersmith
I guess the 18 million dollar recovery.gov website needs a lot of horsepower to run.
http://blogs.abcnews.com/thenote/2009/07/18m-being-spent-to-redesign-recoverygov-web-site.html
i like the way you think:
not resorting to violence => inability to do so => weakness
You clearly have no idea how I think.
I am against American militarism. What I just said was that socialism doesn't fix that problem. The Nazi's were of course also socialists. I didn't even argue against socialism, which may be a good idea in other regards, I just said that it doesn't fix the militarism/fascism problem.
As far as this particular project, the hardware costs are probably not more than $1 million, it will probably cost $5-10 million to design the system, which is justifiable, and then the other $21 million are all administrative costs. Then the project will over-run when the people running the project change their minds halfway through (and then again change their minds back, or just to something completely different), causing the engineering costs to skyrocket, which in turn causes the administrative costs to skyrocket. I wouldn't be all that surprised if this $32 million project ends up costing $70 million. It happens all the time.
The article was pretty weak in info, but $32mil should get you more than $1mil in equipment. Actually, if this is truly could computing you are buying X number of cpu years and Y number of petabytes worth of diskspace with a guarantee of that X will be delivered and that Y will be of some assurance against failure.
Cloud computing is just a segue to computing as a service. Think of it in terms of power, phone, etc. You pay by the use in some manner or another and billed accordingly.
ok, i misunderstood your post. although i wasn't arguing for socialism, i was just pointing out that socialism != having an internationally respected country with a good healthcare system and public education, that's all.
weinersmith
Maybe I read your first post backwards too, sorry.
That's the answer. A government one does. That's what happens when you get to spend other people's money with no accountability to the market.
Think about that for a minute - you have to be 65 to collect, and the average life span is in the upper 70's.
Average life span has been increasing for a while now, and society will adapt. The idea of social security was never to enable people to spend the last 15 years of their life living on benefits. When the 65 years age limit was introduced, the average life span for a man was only a couple of years above that. What will happen now is that the retirement age will gradually increase, and we are probably going to see a unified male and female age of 70 years old by 2025.
no worriez. Lenin teaches forgiveness! (unless you're the royal family)
weinersmith
A government one.
The solution to Social Security is actually very simple. A number of other countries have already done it, including the one I live in. It's just not an idea which would be tremendously popular in the US, and especially not with Libertarians.
Here in Australia, we have mandatory superannuation(that is to say your employer is required to contribute a certain percentage of your wage to a retirement fund of your choice) and the old aged pension(our equivalent of Social Security) is means tested. Rich enough that you don't need it, then you don't get it.
Certainly if you want a more comfortable retirement you should put away additional funds, but the basic rate is enough that hopefully(this hasn't been in all that long) a lot fewer people will need the pension when the time comes. It's good that it's mandatory because it means that people(especially younger people who are less likely to negotiate hard for a 401k) are saving money which will save everyone money later.
The unpopular bit of course is that everyone pays, but not everyone gets any benefit out of it(like most government services really). This pisses off rich people(who don't generally benefit) and libertarians(who generally don't want anyone else to have any of their money), but it works quite well. The system is relatively manageable on our much lower GDP, people who need it get it, it's not going bankrupt, all of that sort of thing.
That's not to say that there aren't plenty of wasteful government projects whose only real goal is to win votes, but Social Security doesn't actually have to be.
We are a country who has been continuously at war for many years. We spend more on national defense than all the other countries put together. We have troops in 130 countries in the world. We have treaties pledging to defend most of them.
We are deeply in debt, and project deficits > $1T for years.
The majority of our Congress is owned by the military-intelligence-industrial complex, Wall Street, police agencies and the prison industry.
Money always can buy power, and otherwise power does what it wants.
So, the idea that we will have mere socialism is silly: We have a fascist version.
"The Constitution, the WHOLE Constitution, and nothing but the CONSTITUTION."
cloud computing = grid computing + virtualization - security - performance
"My life's work has been to prompt others... and be forgotten." --Cyrano de Bergerac
"a small one"....
The question to ask then would be "Does the average lifespan increase, have a corresponding increase in span a person is capable of working?" If people live til 90 on average, but are decrepit at 65, what then?
At least I'm curious if the able to work span increases the same as the life span.
This is not the funny you're looking for.
The average life span in the US is likely to actually start dropping soon because obesity has been steadily increasing and there are many obesity-related or obesity-aggravated diseases (heart disease, diabetes, high blood pressure, etc.) that will work to kill off people early. Life expectancy is climbing partly based on health improvements due to lower smoking rates. Fewer unwanted babies due to legalization of abortion is probably also a factor (they would be more likely to be poorly cared for, often leading to earlier deaths that significantly affect the statistics). However, at some point in the not too distant future, those improvements are going to plateau and the effects of obesity are going to take over.
Diabetes is going to become very expensive for all those private insurers. First they'll try to dodge the costs but eventually they're going to be forced to pick up the tab and employers and employees won't be able to pay when they try to pass the costs on. At that point, faced with a shrinking market and losses (and probably a renewed push for public healthcare) it's going to be their lobbyists against the corn/agribusiness lobbies fighting it out in congress to get healthier prepared foods available for consumers (less high fructose corn syrup for starters). If they were smart, the health insurers would see the writing on the wall and start pushing for that hard now, however odds are they won't actually want to start spending money on this issue until it starts seriously hitting them in the pocket book. It'll probably be too late for about 2 generations of US citizens by the time the insurers start wising up. As for US congressmen doing the right thing by themselves, dead men don't vote or make political contributions. Certainly don't go holding your breath for the Republicans in the cornbelt to do anything.
Laissez lire, et laissez danser; ces deux amusements ne feront jamais de mal au monde. - Voltaire
32M sounds like a giant chunk of change, but, its not even what gets spent on FireFox each year.
You figure a 50 man team of senior devs for a year, and I think that would pretty much do it.
This is my sig.
Social security is going broke because the govt. borrowed money from it when the income was greater than the outgoings. The reason the current generation is paying for the last is to cover the repayments. Maybe if they had left it alone instead of stealing from it there wouldn't be such a crisis. It's not meant to cover more than a tenth of a lifespan, it's meant to supplement the last 10 to 15 years. But people have relied on it, which seeing as the govt. have already spent it elsewhere, was a crucial mistake. And I see no problem with the current generation paying out for the last. After all, everything around you, and your whole upbringing is due to the last generation. That is what civilisation is, looking after your elders, having respect for those who spent so much getting you to where you are now. I presume you think that anybody reaching retirement age should just be euthanised to save you money. I have a counter proposal whereby anybody who doesn't graduate with a decent degree should be killed off. If we are going to spend that much money educating, feeding, protecting and otherwise supporting the younger generation I would rather we got a decent return. Allowing idiots to progress beyond childhood is wasteful.
Yes that last bit was sarcasm, did you notice ?
I don't have kids, yet my wages are taxed just the same, in fact more than parents are. What is _my_ return ? I put money in my whole life but then tight ass newbies want to prevent me from collecting my dues. Just you wait.
Except that they are building their own cloud, not buying cloud infrastructure from a supplier like Amazon. They can then offer scientists time and space (arf) on their own cloud in the same way that Amazon do with the public but without any of the issues of releasing their data onto the internet.
Besides, there are multiple definitions of "cloud computing", and your example is just one of them. Another popular one at the moment is massively parallel data processing systems with huge amounts of storage (i.e. data grids).
'what kind of cloud computing project costs $32 mil' it says.
hell. even the bare costs of the number of servers that would be required to run a cloud of that size would amount to a goodly portion of that $32 mil. EVEN if you buy them in bulk. thats leaving out everything else including the datacenter setup, software, administrators, engineers, the team to create the project.
doh. i suppose cloud computing comes free, in the universe the article poster lives in.
Read radical news here
The Nazis were nationalist, corporatists and social darwinists - fascists. They called themselves socialists - national socialist - but their chief ideological enemy was communist socialism. The Nazis were much closer to fascists than socialists.
Disturbingly, as I read this right now, it's marked "(Score: 2, Informative)"
Where's the "+1: Too Much Informative" mod when you need it?
I'm not saying that just for disinformation, though I do think that Weapons of Mass Destruction are Bad Things - I've worked there on data network consulting applications. When I said not to confuse tool-building with pork-barrel, that was a response to somebody else who said that spending $32M on a cloud-computer was pork-barrel.
While there are a few people at DOE who do graphics or computing just because it's cool, or because almost nobody else who does high-security computing research does it on supercomputers, they mostly view computing as a tool to do science and/or engineering. They're trying to model nuclear explosions, and it takes a lot of computing horsepower to do that.
And the "Imagine What You Could Do With A Cray" poster on their walls was cool, but when there's only one broadband cable system to get data across campus, back when 10Mbps Ethernet was still fast, and a few thousand people imagining what they can do with that Cray, networking problems get really hairy.
But yeah, apparently somebody else had made the mushroom cloud joke a bit earlier than I had. Didn't say anything useful about their computing environment, though.
Bill Stewart
New Fast-Compression-only CPR http://preview.tinyurl.com/dy575ks