Slashdot Mirror

And just how long will it be before someone decides to create a WWG application that uses it as one vast storage pool of copyrighted material with distributed indexing of the contents and the RIAA, MPAA, ... of the world sue the whole thing into non-existance or buy laws to make it a criminal offence to run it?

Not long: http://www.bbc.co.uk/rd/projects/prism/technology.shtml

Structured correctly you wouldn't know who was adding to it or downloading data from it. After all a download would be just be a request to replicate a bit more data making a vastly distributed virtual filesystem a bit more redundant. You may not even be able to tell if it was someone making a request to make a local copy or the software automatically increasing storage redundancy of static data (assuming that there's no logging).

Quite the opposite: http://www.globus.org/toolkit/docs/latest-stable/security/key/#security-key-mutualauthentication

On the one hand, you've cloud computing resources, which supply minimal information, some source but a LOT of buzzwords, versus distributed computing versus grid computing, where there's a lot more information on what is (and is not) provided, and a lot more code is there. Ultimately, the best way to tell if something is worthwhile is to see if the provider thinks it's worthwhile. Cloud providers don't think it worthwhile to do for profit the work grid providers do for free, ergo cloud providers don't rate their own service highly. If they don't, why should anyone else?

http://www.globus.org/

The limitations of transfer rates for scp is often the round trip time that consumes time for confirmation of received packages. This is a serious issue for transfers from the Europe to the US West Coast (around 200 ms) or to Australia (around 400 ms). Having several parallel TCP streams can solve this problem and has been in use for many years for transfer of data in High Energy Physics. An example of such a solution is GridFTP http://www.globus.org/toolkit/docs/4.0/data/gridftp/.

IBM are also heavily investing in Grids, particularly with their support of the Globus Alliance Toolkit (see http://www.globus.org/)

Crazy? No, they are aiming at different targets. Mainframes are controlled by individual companies, grids are hoped to eventually be the equivalent of TCP - ubiquitous, reliable and cheaply available everywhere. That means your next Windows Vista T1000, Ubuntu Beam-me-up (TM) and self-aware toaster will seamlessly provide services to allow you to participate in grid-like problems just like any appliance. No more downloading the google toolbar to help cancer research, another app for cracking encryption, another app for predicting weather and so on...

I wouldn't take the rawstory.com story too seriously.

For one thing, the author seems to think that "Grid" and "TeraGrid" are the same thing. A Grid is a generic term for a set of computer resources, possibly spread across multiple administrative domains, working together using Grid software (such as Globus http://www.globus.org/). The TeraGrid is one specific Grid project.

Beyond that, I don't know why he thinks the Department of Homeland Security has anything to do with this. The TeraGrid is not, as far as I know, used for classified research. Yes, Argonne National Laboratory is a TeraGrid site, and I'm sure they do classified work as well, but there's no reason to assume there's any connection between the two.

The TeraGrid has the same security concerns as any other large computer system: maintaining the integrity of users' data, keeping intruders out, and so forth.

The IT dept should know not to trust "Snake Oil Corp.", however anything from "Citrix Corp" should be fairly safe.
No shit. The problem never was that, but wheter "1024D/40558AC9" should be trusted, or perhaps if "1024D/8E297990" is more trustworthy.

I can label my key with "Citrix Corp" if I like, and I might even be able to convince Verisign to sign it as these guys did.

Code Signing was never intended for [guaranteeing code is safe].
But users don't know that. They're being duped into thinking that if only the path between the software developer and the user weren't so muddy, there wouldn't be any viruses or spyware or adware or whatnot.

Nevertheless, that path is the one attacked less frequently, but it is instead the software that Microsoft got to you completely intact that is the software you cannot trust.

'It is like saying host based IDs or anti-virus are useless, because if you can compromise the system you can turn them off.'
That's exactly why they are useless. You don't do scanning on a machine that you fear might be compromised. You do it from a machine that you know isn't, but that still has access to the other machine's frozen state.

Support contracts are available, but not mandatory.

Not affiliated, just a happy customer.

Take a pinch of Standard Linux
Wrap it up in Xen
Add a touch of SELinux
And a little bitty bit of Globus
Oh like a Sandboxed Platform
Oh Lordy, Lordy, mixed with Free and Open Source Code
You know you lump it all together
And you got a recipe for a Multi Vendor Development scene
It is coming though, you know, you know.

What we have is a great big melting pot
Big enough enough enough to take every vendor and all IT's got
And keep it stirring for a hundred years or more
And turn out Application Service and Content Providers by the score.

With apologies to Blue Mink .

http://www-unix.globus.org/toolkit/docs/3.2/index. html "Globus Toolkit 3.2 Key Concepts (not yet available)"

If you want working single-sign on across domains/organisational broders you can use Globus, the defaco open-source Grid framework/toolkit mostly in use in the research, univeristy world.

New reworked version 4.0 scheduled for release this spring will provide for authorization thru firewalls and webproxies using Web Services tech.

http://www-unix.globus.org/toolkit/docs/developmen t/4.0-drafts/GT4Facts/index.html/
http://www-unix.globus.org/toolkit/docs/developmen t/4.0-drafts/security/authzframe/index.html/
http://www-unix.globus.org/toolkit/docs/3.2/index. html#security/
http://www.globus.org/

If you want working single-sign on across domains/organisational broders you can use Globus, the defaco open-source Grid framework/toolkit mostly in use in the research, univeristy world.

New reworked version 4.0 scheduled for release this spring will provide for authorization thru firewalls and webproxies using Web Services tech.

http://www-unix.globus.org/toolkit/docs/developmen t/4.0-drafts/GT4Facts/index.html/
http://www-unix.globus.org/toolkit/docs/developmen t/4.0-drafts/security/authzframe/index.html/
http://www-unix.globus.org/toolkit/docs/3.2/index. html#security/
http://www.globus.org/

If you want working single-sign on across domains/organisational broders you can use Globus, the defaco open-source Grid framework/toolkit mostly in use in the research, univeristy world.

New reworked version 4.0 scheduled for release this spring will provide for authorization thru firewalls and webproxies using Web Services tech.

http://www-unix.globus.org/toolkit/docs/developmen t/4.0-drafts/GT4Facts/index.html/
http://www-unix.globus.org/toolkit/docs/developmen t/4.0-drafts/security/authzframe/index.html/
http://www-unix.globus.org/toolkit/docs/3.2/index. html#security/
http://www.globus.org/

If you want working single-sign on across domains/organisational broders you can use Globus, the defaco open-source Grid framework/toolkit mostly in use in the research, univeristy world.

New reworked version 4.0 scheduled for release this spring will provide for authorization thru firewalls and webproxies using Web Services tech.

http://www-unix.globus.org/toolkit/docs/developmen t/4.0-drafts/GT4Facts/index.html/
http://www-unix.globus.org/toolkit/docs/developmen t/4.0-drafts/security/authzframe/index.html/
http://www-unix.globus.org/toolkit/docs/3.2/index. html#security/
http://www.globus.org/

They are talking about "Fast" TCP, which AFAIK just consists of a better routing algorithm and using multiple TCP streams at once.

No, FAST has nothing to do with routing or multiple streams.

(Users of FAST may also be users of multiple streams for data transfer, as in GridFTP.)

FAST (follow the link in the parent) is one of many approaches to improving the throughput of a stream by changing the algorithms and timing for packet transmission.

My MSc thesis implemented a grid service for (non-computer science) computational scientists through Globus (V3.2) that harnessed a Condor (V6.6) backend for a heterogeneous computational grid. Here at the university, because funds are not exactly forthcoming, we utilize idle resources in computer labs and classrooms.

That's a supercluster, not a grid. The Grid is a type of organizational system which brings together resources; but does not necessarily imply high throughput or latency on computation.

Computational Grid is working out the logistics, and has a software package that'll get you on the grid.

Check out these links for details, there is a lot of work in this area. Comparing this to @Home type projects is the wrong way to go... each node on the grid can be a gatekeeper to a cluster, where a parallel job is run. The problems involved are more complex than what a centralized server/organization can take care of.

http://www.globus.org//
http://www.ggf.org//
http://www.globusworld.com//

Some details.. much more out there.
http://www.grids-center.org/news/clusterworld//
http://www-106.ibm.com/developerworks/grid/library /gr-design.html/
http://www.casa-sotomayor.net/gt3-tutorial//

Plus, lots of academic papers out there.. this is a pretty interesting subject.

Globus provides an implementation of OGSA/OGSI and soon WSRF, all protocols developed for grid computing. The Global Grid Forum meets regularly (at conferences) to decide on standardization of grid protocols.

Note that just because the grid protocols are being standardized does not mean anyone can have access to one "giant grid". It just means that heteregeneous sites are able to interoperate, authentication/authorization control is up to the participating sites' local policies and via virtual organizations which can be (simply) thought of as aggregates, the 'master' groups (but VOs can be much less coarse than that).

There is a lot of advanced work being done in this area, see:

http://www.globus.org/
http://www.ggf.org/

Cyc is a joke. Its only purpose is to suck down funding from government contracts.

Why does mac have to go out and roll-their-own clustering software when it would be a lot more helpful to embrase something like Globus which is being adopted for grid computing world wide.

It would fit right in as it can run on top of JBoss and prefers a *nix environment.

Either that or OpenMosix... but not another new one.

The article mentions "The five-year DES hopes to generate about 100 terabytes of data" that will be released to the public at regular intervals....

This kit is probably one example of why the world needs more 92 Tbs routers; sharing the data generated by this baby will probably be a task not unlike that faced by the Large Hadron Collider at CERN. You're going to have to have a really nice architecture and set of protocols to be able to efficiently pass around these images - possibly this is where Grid Technology comes in to play....

Of course, then you'll need something to actually process the images on! I guess Intel and AMD still have a rosy future ahead of them...

I agree with what others have said Grid Computing is distinct from other forms of distributed computing.
Check out OGSA - the Open Grid Services Architecture - and learn what is and is not a Grid. This is the de facto standard for building Grids.
Even new products that are *sold* with Grid in their title aren't necessarly THE GRID though they might be A GRID.

"Grid" as a concept is mostly just a buzzword. Oracle10g is a good example.

But Grid as a standard (under development by the GGF, OASIS and others) is something a lot more specific. What that emerging standards-compliant Grid offers is:

1. End-to-end encryption, based on certificates and public key
2. Virtual organizations, in which there is a closed "community" of systems and their users who can participate in a particular Grid (somewhat VPN-like)
3. Event-driven framework, instead of client/server or push/pull models -- this is a major win for some applications

The Globus toolkit is one messed up pile of confused & confusing software (yes, I'm running 2.4 and 3.0 and develop for it). But it will change a lot for the better over this upcoming year, if it keeps to schedule, to build on Web services (Tomcat, etc.) rather than re-inventing WS+Grid. Adding the points 1-3 above to WS will work a whole lot better, I think/hope, than re-creating most of WS in the Grid services (GS) Globus toolkit.

Bottom line: The vision/plan of standards-compliant Grid computing does offer some real advantages and promise. It's not for every application, every user and every organization, any more than, say, cluster or distributed computing is. But today's Globus and other Grid standards are in pretty early stages, and only barely useful for real tasks (i.e., see how the Teragrid fell victim to fairly mundane attack).

This sounds similar to what some people (such as Entropia) are attempting on "The Grid" (see The Globus Project for more wiffly utopian detail on that).

Of course, it also leads to problems. Like security - how much can you trust an autonomous node to return correct data?

To what extent is there scope for malicious nodes deliberately returning incorrect results? And given the autonomous nature of the nodes, they can turn on and off (crash, be turned off) at any time and their performance will fluctuate depending on what other load (that you can't control) is placed on them, so you may want checkpointing and/or redundancy going on.

Which leads to the question of how to co-ordinate it all to maintain consistancy should you wish to roll back. All very app-specific, to be sure, but it's the detail that prevents stuff like this from being used by more people.

Blah!

GridFTP

Oddly enough, WSDL and SOAP are mentioned, but never really discussed. And the would be probably better suited than CORBA.

WSDL and SOAP are used in version 3 of the Globus Toolkit, which is a well-known open source grid toolkit. In that version, all grid services are Web Services.

Oddly enough, WSDL and SOAP are mentioned, but never really discussed. And the would be probably better suited than CORBA.

WSDL and SOAP are used in version 3 of the Globus Toolkit, which is a well-known open source grid toolkit. In that version, all grid services are Web Services.

The real and specific problem that underlies the Grid concept is coordinated resource sharing and problem solving in dynamic, multi-institutional virtual organizations. The sharing that we are concerned with is not primarily file exchange but rather direct access to computers, software, data, and other resources, as is required by a range of collaborative problem-solving and resource brokering strategies emerging in industry, science, and engineering. This sharing is, necessarily, highly controlled, with resource providers and consumers defining clearly and carefully just what is shared, who is allowed to share, and the conditions under which sharing occurs. A set of individuals and/or institutions defined by such sharing rules form what we call a virtual organization (VO).

Well, the article was about grids, not p2p systems.

Here's one toolkit for creating grid programs with java.

Personally, I just don't see grid computing work where you ship your stuff out to 3rd party computers. There is the network latency, and the security aspects. But it might work for a company to maintain their own grid. That I could see. Maybe.

This may be great for a few high profile applications that users are willing to support. But the Globus Toolkit OGSA project has higher ambitions OGSA and arguably a better chance of making a difference in the next generation of the WWW.

CxC is a closed, expensive software platform written by one company. Everything that open source software exists to prevent. Lets see an OpenGridChallenge and lets not make it War. Lets make it competitive, but make it a constructive challenge. Like an Earth Weather simulator, Maintaining the structure of an underwater skyscraper built with proactive energized joints, or a biological process simulator. The project with the most efficiency in some given resepect wins.

Globus is an open-source grid computing project based on the Open Grid Services Architecture. Start with that.

CxC is a closed, expensive software platform written by one company. Everything that open source software exists to prevent. Lets see an OpenGridChallenge and lets not make it War. Lets make it competitive, but make it a constructive challenge. Like an Earth Weather simulator, Maintaining the structure of an underwater skyscraper built with proactive energized joints, or a biological process simulator. The project with the most efficiency in some given resepect wins.

Globus is an open-source grid computing project based on the Open Grid Services Architecture. Start with that.

This is not a supercomputer. It is just a collection of individual laboratories and universities running plain old batch systems on Beowulf (or beowulk-like) clusters. To transfer jobs and data between sites they're using Globus. It's really not interesting stuff, certainly not newsworthy. This Grid is held together with duct tape and spit. The Globus software is woefully immature (Expect complete rewrites years. Expect refusal to fix crippling bugs because the next complete rewrite due out next year might fix it.) The Globus software is only a partial solution, it really only cares about moving data and jobs between sites. Anything more complex like automatically assigning jobs to the least busy site, distributed management of user accounts, or even ensuring that jobs eventually finish requires piling more software on to. This additional software is also constantly changing. The result is a system that is at best frustrating to install and configure, assuming someone else has assembled a known working package. If you're doing it by hand, expect days of effort. When things go wrong the system is a nightmare to debug as the many, many layers of software don't propogate errors (that is, when the error is logged at all, sometimes they'll just silently fail).

Links:

• Globus FAQ
• Python Globus (pyGlobus)
• The Condor Project
• Beowulf FAQ

Links:

• Globus FAQ
• Python Globus (pyGlobus)
• The Condor Project
• Beowulf FAQ

Since the article doesn't really have to do with grid computing. Here are some real Grid Computing links.

Globus Toolkit
LSF
openPBS
gridengine
OSCAR
ROCK MPP
maui

and last but not least: beowulf cluster
---

We've all heard the new buzzword, "grid computing" quite a bit in the news recently.

This is not an example of the Grid Computing (ala Globus) that we've been hearing about. This is another example of laying out processor cores on a chip. So a better thing would be to compare this to the ideas for the UltraSPARC V and IBM BlueGene computers where multiple processing cores are put on one chip and then arranged in a grid (think physical grid) architecture.

Grid Computing deals with computation and information sharing seemlessy across a network, they used to always say like how the power grid works. Which in reality is about right as it doesn't always work as advertised.

Anyway, Grid Computing is mainly concerned with software to allow multiple computers to work together seemlessly. This includes registry services, single sign of, information transfer, etc.

This appears to be the rather fortunate result of a phenomenon called "Buzzword collision", where two different projects pick the same buzzword in hopes to really confuse people who don't read the articles and trick PHBs into thinking that each project is ueberimportant.

The ad closes with the slogan "Linux. The Future is Open. IBM."

My jaw is wide open.
Ok, IBM wants to capitalize on Linux, IBM is a company, in the past IBM has been a corrupt monopoly trying to stranglehold their clients with the proprietary ties after fscking them in the first place.

But the memes passed by this particular advertisement are something radically different than those teached by conventional advertisements and pro-capitalist ethos (consume! consume! don't think about tomorrow! spend now!).

Really, this stuff is jaw-dropping.
It's not like the cluetrain stopping in front of IBM (now we only need an express train passing in Darl McBride's office like that Athlon vs Pentium III commercial), but it's like someone saying that the time is mature for an economy based on Free Software to be born and TRY to impose herself on a wide-open scale.
Not Eazel Gnome Nautilus and crap like that, not the dotcomboom of 121 different Linux distros, but a wide world collaborating on making better, documented, free, opensource and secure systems with minor tweakings among them just to make sure that they suit individual needs. (2414 different Gnu/Linux distros!!! ;D)

Hell, also Microsoft is committed to a large-scale opensource initiative.

And I also think that this particular advertisement is what we waited in order to say that GNU and Linux have won.

+ + + +
BSD, on the other hand, is dead... :D (ok, that was a joke)

There is also a direct trade-off between more general purpose systems and systems custom tailored to a task. Good examples are Deep Blue and Blue Gene. Both of these systems are designed with a particular task in mind (i.e. chess and protein folding) and therefor are able to leverage knowledge about the problem space to constrain the kind of hardware, the particular low-level instructions and the information flow within the system while achieving signifigantly greater performance on a small class of problems. I work with clusters that are used in scientific communities that have various researchers working on various problems. In these cases, the questions are about basic applicability of a particular problem to a particular architecture. For example a cluster with high-speed interconnects made of good COTS hardware will allow a user with a very granular problem to effectively use the cluster and it will also allow a user who needs the high speed interconnect because the problem space demands a high degree of internal communication. But the first researcher might also be able to make use of a grid of (for instance) many more computers with a total lower cost because (s)he doesn't need the high speed interconnect. The Earth Simulator gains a lot of performance (on a class of problems) because of the underlying vector processor architecture. Given the right internal bus it is conceivable that adding vector processor daughter boards to the next generation of COTS clusters could achieve similar results--but, of course, only for problem spaces that make efficient use of such processors and aren't bottlenecked by the communication requirements.

Real answers are always more complicated. For example: the equations needed for nuclear simulation will probably require dedicated hardware (as the need for protein folding has lead to Blue Gene) to achieve the results that the Pentagon needs. But for many super computing tasks, the flexibility of COTS clusters will still be compelling, especially for areas where the algorithms are not yet fully developed (e.g. brain simulation). An interesting keynote at OLS 2003 argued that (some of) the problems are not going to be the local computing power but the need to move large quantities of data between research labs across the world and combine computational systems using the 'grid.' (For a down home examples of problems that have been successfully tackled through course granular distribution just look at SETI@Home and Distributed.Net. So its not just the flops anymore...

No, you can't. Just try finding an ISP that would risk disrupting their systems to let you do that. The promise of the Grid is that it creates a standardized method of sharing computing resources, with a rich security model, so that ISPs could deploy grid software on their systems and know that grid applications are well-behaved. In the Globus Toolkit framework, for example, all grid applications run within an application server container, such as Tomcat, which means they're in a Java sandbox and can't muck with things they're not supposed to.

A major collaborator on this stuff is Globus which provides an API for grid applications. Same people who are partners with IBM in the butterfly.net game grid.

Maybe MTU can use it to store their students' Kazaa archives.

------

Damn, the mouse slipped and I hit submit. That link should have been the Globus Project...

a bit off topic I realize, but just the thought of possibilities of creating virtual worlds much like someone would mod a game just makes me want to dive right into my simulation books... well right after I play a bit more MOO III.

I've seen a ton of questions asking what Grid computing is. The most common one being how does it differ from parallel/distributed computing?

First off, I highly suggest reading The anatomy of the Grid by Ian Foster et. al. It provides a pretty good overview into this whole Grid thing.

But for the lazy, here's a little bit. The Grid is more than parallel computing. Typically with parallel/distributed computing the problem or resources are static or both. Grid allows both of these to change. In a nutshell, Grid computing means not having to worry about where the compute resources are. Just start a calculation and it gets done. Just like how you don't worry where your power comes from, you just plug in.

The core of the Grid is virtual organizations. Under a VO, I could get together with a few friends and pool our resources. We could set up a registry and some factories (I'm speaking OGSA here, but whatever) and create some certificates. Then, we could submit jobs to the Grid and not have to worry about the resources that they're running on.

GSI provides some really nifty security features (based on X.509 I believe). Basically you provide a mapping that allows other authorized users to run commands on your computer. When you're on the Grid you create a proxy for your certificate that is passed to the process that you run on this other computer. Then if that computer needs more resources, it can create another proxy certificate and delegate to another server.

Also, Grid computing is more than just computing. There is data storage and instrumentation sharing also. You might want to check out PPDG, GriPhyN and TeraGrid for examples of these systems.

If you're interested in playing with the GRID, you can go download Globus Toolkit 3.0 Alpha or the Java CoG Kit which is a pure Java implementation of Globus 2.x (it's much easier to install than the regular Globus 2.2.x).

I've seen a ton of questions asking what Grid computing is. The most common one being how does it differ from parallel/distributed computing?

First off, I highly suggest reading The anatomy of the Grid by Ian Foster et. al. It provides a pretty good overview into this whole Grid thing.

But for the lazy, here's a little bit. The Grid is more than parallel computing. Typically with parallel/distributed computing the problem or resources are static or both. Grid allows both of these to change. In a nutshell, Grid computing means not having to worry about where the compute resources are. Just start a calculation and it gets done. Just like how you don't worry where your power comes from, you just plug in.

The core of the Grid is virtual organizations. Under a VO, I could get together with a few friends and pool our resources. We could set up a registry and some factories (I'm speaking OGSA here, but whatever) and create some certificates. Then, we could submit jobs to the Grid and not have to worry about the resources that they're running on.

GSI provides some really nifty security features (based on X.509 I believe). Basically you provide a mapping that allows other authorized users to run commands on your computer. When you're on the Grid you create a proxy for your certificate that is passed to the process that you run on this other computer. Then if that computer needs more resources, it can create another proxy certificate and delegate to another server.

Also, Grid computing is more than just computing. There is data storage and instrumentation sharing also. You might want to check out PPDG, GriPhyN and TeraGrid for examples of these systems.

If you're interested in playing with the GRID, you can go download Globus Toolkit 3.0 Alpha or the Java CoG Kit which is a pure Java implementation of Globus 2.x (it's much easier to install than the regular Globus 2.2.x).

I've seen a ton of questions asking what Grid computing is. The most common one being how does it differ from parallel/distributed computing?

First off, I highly suggest reading The anatomy of the Grid by Ian Foster et. al. It provides a pretty good overview into this whole Grid thing.

But for the lazy, here's a little bit. The Grid is more than parallel computing. Typically with parallel/distributed computing the problem or resources are static or both. Grid allows both of these to change. In a nutshell, Grid computing means not having to worry about where the compute resources are. Just start a calculation and it gets done. Just like how you don't worry where your power comes from, you just plug in.

The core of the Grid is virtual organizations. Under a VO, I could get together with a few friends and pool our resources. We could set up a registry and some factories (I'm speaking OGSA here, but whatever) and create some certificates. Then, we could submit jobs to the Grid and not have to worry about the resources that they're running on.

GSI provides some really nifty security features (based on X.509 I believe). Basically you provide a mapping that allows other authorized users to run commands on your computer. When you're on the Grid you create a proxy for your certificate that is passed to the process that you run on this other computer. Then if that computer needs more resources, it can create another proxy certificate and delegate to another server.

Also, Grid computing is more than just computing. There is data storage and instrumentation sharing also. You might want to check out PPDG, GriPhyN and TeraGrid for examples of these systems.

If you're interested in playing with the GRID, you can go download Globus Toolkit 3.0 Alpha or the Java CoG Kit which is a pure Java implementation of Globus 2.x (it's much easier to install than the regular Globus 2.2.x).

The feature's usually turned off for security reasons. The only variations on FTP where it is turned on that I've heard of have had an absolutely vicious encrypted authentication protocol on top.