The film is not the finest representation of the project. It's nice, but I winced when I heard that line - what amounts to a jobs program for physicists.
The reality is that there are a number of good reasons to be doing this. There are an enormous number of tech spinoffs that result (you're using one of them). Medical, industrial, informatics, etc - we're solving problems (out of necessity) that the rest of the world hasn't even run into yet. The data rate from one detector is greater than every human being on Earth having 20 phone conversations at once.
We're one of the reasons that the internet was developed to its present form.
But mostly that's good for telling politicians why to fund us, so they can do cost-benefit analyses with Beltway bandits and justify the expenditure to the OMB without being scalped. The real reason for all this is
We Are Not Human Beings If We Don't Explore.
We become sheep. We surf the web and watch network TV and do stuff that is fun but stagnant. Or stuff that is not fun and even more stagnant.
Poking at the fundamental levels of our knowledge is quite different from Googling the result - and takes time, money and expertise. These questions we're asking right now - we're asking them because we hunger for the real story. Fortunately, it's relatively cheap to do so. 5 billion in national terms is the price of a nice dinner in personal terms. In international terms, it's chump change. We'd do it cheaper if we could - but it's hard to examine things a octillion time smaller than you.
We pay it - though there are worthy causes that could benefit from that cash - because succumbing to stagnation is to deny who we are, to turn our backs on the contributions of the giants on whose shoulders we stand, and to declare as a civilization that we're done looking forward - we're happy with what we are now. We roll over and go to sleep.
This one is running at ~5 billion, spread across a number of countries.
The SSC was a classic case of physicists trying to be politicians. The cost would have been much lower if it had been built on the same infrastructure as Fermilab or another existing installation. Instead, those pushing the project took advantage of various national figures' soft spot for Texas, and decided to build there.
Things at that size level are terrifying and bizarre. The timescale and size differentials between them and us insulate us from that fact, however. There may be black holes produced (my officemate was trying to determine what their signature in the ATLAS detector would be), but you couldn't get an atom entire into their event horizon. They're "lost in space" and can't grow - and kill themselves off with Hawking radiation very fast.:) We think.
'Course, there's only so much you can say about things you've never seen. Physics would have to be a whole lot different to be a problem, though. Remember that a black hole with the Earth's mass would have a guaranteed catch radius of about 7 mm. Gravity is VERY weak (10^41 times weaker than electromagnetic... ten thousand billion billion billion times weaker) - if something strange were to occur, I'd expect it to occur with one of the other forces.
Nope. Ed is a high-energy physicist. I'm here too - and on OS X.
Excel is not within orders of magnitude of the number-crunching capacity we need here. Nothing like that in physics? Try petabytes-per-year data rate, and on-the-fly data reduction and analysis. After data reduction. The raw data rate out of the detector will be on the order of 100 TB/second, but we cut that by a factor of 10^6 or so. Right now, in preparation phase, I still handle terabyte data sets, trying to figure out what's going to happen when the detector turns on.
R is also nice. After you've done the real data reduction.
You're not real familiar with normal english usages, are you? It's hard to imagine the term being more diluted.
Most english speakers of median intelligence are capable of separating meanings by context. Thus, a "torturous" math test is significantly less laden with horror than "torture" at Abu Ghraib (under US management), which is in turn much less nasty than "torture" at Abu Ghraib (under Saddam's management).
Sounds like a case of political newspeak overwhelming reality.
Make sure you don't have any slave hard drives at home or work, either. Dilutes the meaning.
I guess I would disagree that the Grid is where HTTP was circa 1993. Whereas there was just one WWW and it was based on one protocol, the present definition of Grid computing is hazier.
There are several competing definitions of "Grid" going around - from the happy-big-cluster idea that Apple calls Xgrid (bad name, good product, IMHO) to the TeraGrid and NCSA grids in the US to the LCG/Grid3/Nordugrid to SETI@home. They all speak different languages and are built on different models.
Most of the definition problems come from the degree of heterogeneity involved in the Grid in question. Obviously, Xgrid runs only Macs.:) TeraGrid runs on a very specific set of gov't -owned CPUs of a limited family of processors - basically, Itaniums. LCG/Grid3 are a bit less homogeneous - a selection of versions of RedHat. NorduGrid is very diverse, Linux-wise. SETI@home is positively promiscuous.
Now, have a look at the range of software each can run. SETI has ONE program available. Its wild heterogeneity makes it tightly limited in a resource-limited development environment. Xgrid can run almost anything a Mac can do. This is the tradeoff - as you support more platforms, it becomes harder to support a generality of packages on each platform.
If you make your code extremely light and portable, you can afford to push the code and compile at runtime, or do relatively frequent recompiles and updates on the known sites. This is hard to scale, though.
Another problem is managing the sites in question. Again, scaling. With 50 sites, you know the sysadmins and can interact with them on run issues and security. With 5000, there is no chance. You must have automated checks and maintainence. This is also nontrivial.
A number of solutions are being tried to these problems, as well as to security, load-balancing and storage optimization. They must be solved in the near future, and things look good for that. The most common solution, AFAIK, is the GLOBUS grid middleware - it standardizes a lot of this stuff. It is imperfect and needs work - but it's coming along. Previous comments that Globus' UI is imperfect are silly - it's like carping about the UI of a machine-code instruction. Others middlewares also exist in some form or other, and a few will eventually emerge as solutions to the various problems. Again, the solution you use will vary according to the problem you face.
Eventually, the goal is to have a fairly portable, secure, flexible framework that will run a reasonable number of applications on a reasonable number of platforms. The software has to be admin-friendly - no need for root on the boxes, easy to set up, easy to remove. These are all adjustable requirements.
Right now I run massive high-energy physics software (preinstalled on the site) on tens of sites in the US and Korea. It's not user-friendly - it's in development. It's extremely powerful, even now - my jobs are gargantuan and the disk to store them even more so. I need only one application set, and I have it. In the future, things will be easier - this is all in development mode. However - it DOES work, it WILL continue to improve, it continues to become more secure, and someone (when things are ready) will code up an interface to it that will make it friendly.
That's pretty good. Naysayers take note. This isn't a vaporware idea - it's just a difficult problem with a lot of blanks left to fill in.
DNA with M&M's. You'd have to have a planet's worth of M&M's to do anything useful.
Seriously - I kept watching because I was sure it would eventually get better. Right?.......
Never. Fantastic movie. Loved the pasa double. Hilariously funny.
Of course, you have to have a certain sense of humor. Cold Comfort Farm and O Brother Where Art Thou fall roughly in the same bin.
The comment above shows that the site wasn't read/understood before posting.
The projector (both in stills and on the avaliable film) compensates for both surface color and configuration automatically and applies the transformation on a pixel-by-pixel basis. Good idea, IMHO. Sure seems to work. Seems to be arbitrarily flexible, to first order.
There are definitely artifacts, but they seem to follow curved edges - not surprising when one realizes that pixels are finite and rectangular.
I wonder how the reflectivity of the surface affects the adaptation.
Watch the movie - it's pretty clear on the process.
I have an old Handspring Visor, and I keep roughly 250 books on it (memory card). I use it primarily as a reading device, on the order of 2 books a week.
Best software I ever tried for reading is Weasel Reader, hands down. Mentioned further down.. look into it. zLib compression halves the size of your book, and on-the-fly decompression is very fast. And it's open source. It also lets you rotate your text into landscape or reverse-landscape - very convenient reading format.
Files are generated from text using a small linux program called makeztxt. I've built scripts around it to clean up, edit and compress all of my text files, and keep them organized. All of this is doable on OS X as well.
Get a memory card. Yeah, I know it's not cheap. Get one that matches your digital camera so it can serve double duty. Put all your books on that - you'll hate having to sync new books on (slow) and having to go back to your computer to get another one when you finish. Especially if you accidentally hard-reset. At the airport.:) Keep your whole library with you. Have a selection.
Don't get a non-backlit screen unless you have to. Color isn't necessary, but non-backlit screens are very hard to read in twilight conditions, even when you turn on the 'glowing text'-type lighting.
Resolution doesn't matter. You can read the letters just fine in 160x160, and the higher-res screens _seem_ to have less real estate - thus, smaller letters. The backlit screens are definitely smaller. 'Course, it's easier to see a small letter on a luminescent background than a larger letter on a gray background. Then again, backlights chew battery mercilessly.
No kidding. Especially since it sounds more like a renderfarm than a single "supercomputer".
Then there's the data storage. That's nothing (though I say it myself). Go to high-energy physics for serious storage/processing centers. Little ol' me uses ~15 TB from 4 months of work, and I'm nothing compared to some people's requirements. I use up about 0.4% of our mass storage here (rough guess).
WETA's cool - but not in supercomputer/HPC land.
I'm working with Grid and ftsh as we speak. I'm a physicist, not a professional coder. I write reasonable code, but I'm no purist. With that said...
ftsh has great utility in the realm it's written for. Obviously, it's not a basis for installing kernels or doing password authentication. In a Grid (not just distributed) environment, things break for all sorts of reasons all the time. You're dealing with a Friendly Admin on another system, one who may well be unaffiliated with your institution, project or field of study. He doesn't have any particular reason to consult with you about system changes.
Now you find yourself writing a grid diagnostic or submitter or job manager. One does not need strongly typed compiled languages for this. Shell scripts are almost always more efficient to write, and the speed difference is unimportant. Right now, most Grid submitters are being written in bash or Python or some such. Bash sucks for exception handling of the sort we're talking about. Python does better with its try: statements, but there's room for improvement. ftsh is a good choice for a sublayer to these scripts. One writes some of the machinery that actually interacts with the Grid nodes and supervisors in this easy, clear and flexible form.
Now there are a lot of specific points to answer:
One needs a Windows port to be able to make the Grid software we write in Linux available to the poor drones who are stuck with Win boxes.
This is not a code spellchecker or coding environment. At all.
This is not a crutch for inadequate programmers. This is a collection of methods to deal with a specific set of recalcitrant problems.
As I was pointing out before, this is, after all, an unstable system. One is using diverse resources on diverse platforms in many countries at many institutions. I appreciate the comment made by unixbob about operating in heterogeneous environments.
This isn't a substitute for wget. One uses wget as an example because it's clear.
The "pull" model breaks down immediately when there is no unified environment, as is described on infrastructures.org. When you're not the admin, and your software has to be wiped out the minute your job is done, "push" is the only way to do it. This is the case with most Grid computing right now (that I know about)
All the woe and doom about the sloppy coding and letting the environment correct your deficiencies is... ill-thought-out. That's what a compiler is, folks. Should we all be coding in machine language?:) Use the right tool for the job and save time.
I do agree, however, that one should indeed hone one's craft. Sloppy coding in projects of importance is inexcusable (M$). There is no reason to stick to strict exception handling, however, in the applications being discussed by ftsh's developers (the same folks who brought you Condor). When code becomes 3/4 exception handling, even when the specific exceptions don't matter, there's a problem, IMHO.:)
Here's a recent overview article on the status of Higgs in the LEP data (refinement and rehashing of stuff that's not really new anymore). Go to http://arxiv.org/PS_cache/hep-ph/pdf/0402/0402231. pdf
The total LEP experiment sigma comes out as less than two for a 115 GeV SM Higgs. That's not compelling. However, some VERY nice "gold-plated" 4-jet events were seen in the ALEPH detector, and it seems like there's a good chance that 115 GeV will be a good place to look in LHC.
Speaking of LHC, here's a webcam that lets you look at the ATLAS detector being built.:)
http://atlaseye-webpub.web.cern.ch/atlaseye-webpub/web-sites/pages/UX15_webcams.htm
Hm. Go learn some of the math involved, and come back when you understand that there really are some compelling reasons for Higgs to come into the picture. Or are you aware of something that's been overlooked? You have a good reason that photons are massless and W/Z bosons aren't? Can you tell me why electrons weigh less than taus? Can you tell me how "mass" comes about?
That plus the fact that the possibilities include standard model Higgs and SUSY Higgs, light Higgs, heavy Higgs, MSSM doublet Higgs, all in different variations...
We didn't ask for all these particles to show up. We're just trying to figure out what we're seeing in nature.
Slashdot Mirror
The film is not the finest representation of the project. It's nice, but I winced when I heard that line - what amounts to a jobs program for physicists.
The reality is that there are a number of good reasons to be doing this. There are an enormous number of tech spinoffs that result (you're using one of them). Medical, industrial, informatics, etc - we're solving problems (out of necessity) that the rest of the world hasn't even run into yet. The data rate from one detector is greater than every human being on Earth having 20 phone conversations at once.
We're one of the reasons that the internet was developed to its present form.
But mostly that's good for telling politicians why to fund us, so they can do cost-benefit analyses with Beltway bandits and justify the expenditure to the OMB without being scalped. The real reason for all this is
We Are Not Human Beings If We Don't Explore.
We become sheep. We surf the web and watch network TV and do stuff that is fun but stagnant. Or stuff that is not fun and even more stagnant.
Poking at the fundamental levels of our knowledge is quite different from Googling the result - and takes time, money and expertise. These questions we're asking right now - we're asking them because we hunger for the real story. Fortunately, it's relatively cheap to do so. 5 billion in national terms is the price of a nice dinner in personal terms. In international terms, it's chump change. We'd do it cheaper if we could - but it's hard to examine things a octillion time smaller than you.
We pay it - though there are worthy causes that could benefit from that cash - because succumbing to stagnation is to deny who we are, to turn our backs on the contributions of the giants on whose shoulders we stand, and to declare as a civilization that we're done looking forward - we're happy with what we are now. We roll over and go to sleep.
I stand for something better.
This one is running at ~5 billion, spread across a number of countries.
The SSC was a classic case of physicists trying to be politicians. The cost would have been much lower if it had been built on the same infrastructure as Fermilab or another existing installation. Instead, those pushing the project took advantage of various national figures' soft spot for Texas, and decided to build there.
Now they grow mushrooms in the unfinished tunnel.
Things at that size level are terrifying and bizarre. The timescale and size differentials between them and us insulate us from that fact, however. There may be black holes produced (my officemate was trying to determine what their signature in the ATLAS detector would be), but you couldn't get an atom entire into their event horizon. They're "lost in space" and can't grow - and kill themselves off with Hawking radiation very fast. :) We think.
'Course, there's only so much you can say about things you've never seen. Physics would have to be a whole lot different to be a problem, though. Remember that a black hole with the Earth's mass would have a guaranteed catch radius of about 7 mm. Gravity is VERY weak (10^41 times weaker than electromagnetic... ten thousand billion billion billion times weaker) - if something strange were to occur, I'd expect it to occur with one of the other forces.
Excel is not within orders of magnitude of the number-crunching capacity we need here. Nothing like that in physics? Try petabytes-per-year data rate, and on-the-fly data reduction and analysis. After data reduction. The raw data rate out of the detector will be on the order of 100 TB/second, but we cut that by a factor of 10^6 or so. Right now, in preparation phase, I still handle terabyte data sets, trying to figure out what's going to happen when the detector turns on.
R is also nice. After you've done the real data reduction.
Big difference. :)
Goodness.
You're not real familiar with normal english usages, are you? It's hard to imagine the term being more diluted.
Most english speakers of median intelligence are capable of separating meanings by context. Thus, a "torturous" math test is significantly less laden with horror than "torture" at Abu Ghraib (under US management), which is in turn much less nasty than "torture" at Abu Ghraib (under Saddam's management).
Sounds like a case of political newspeak overwhelming reality.
Make sure you don't have any slave hard drives at home or work, either. Dilutes the meaning.
I guess I would disagree that the Grid is where HTTP was circa 1993. Whereas there was just one WWW and it was based on one protocol, the present definition of Grid computing is hazier.
:) TeraGrid runs on a very specific set of gov't -owned CPUs of a limited family of processors - basically, Itaniums. LCG/Grid3 are a bit less homogeneous - a selection of versions of RedHat. NorduGrid is very diverse, Linux-wise. SETI@home is positively promiscuous.
There are several competing definitions of "Grid" going around - from the happy-big-cluster idea that Apple calls Xgrid (bad name, good product, IMHO) to the TeraGrid and NCSA grids in the US to the LCG/Grid3/Nordugrid to SETI@home. They all speak different languages and are built on different models.
Most of the definition problems come from the degree of heterogeneity involved in the Grid in question. Obviously, Xgrid runs only Macs.
Now, have a look at the range of software each can run. SETI has ONE program available. Its wild heterogeneity makes it tightly limited in a resource-limited development environment. Xgrid can run almost anything a Mac can do. This is the tradeoff - as you support more platforms, it becomes harder to support a generality of packages on each platform.
If you make your code extremely light and portable, you can afford to push the code and compile at runtime, or do relatively frequent recompiles and updates on the known sites. This is hard to scale, though.
Another problem is managing the sites in question. Again, scaling. With 50 sites, you know the sysadmins and can interact with them on run issues and security. With 5000, there is no chance. You must have automated checks and maintainence. This is also nontrivial.
A number of solutions are being tried to these problems, as well as to security, load-balancing and storage optimization. They must be solved in the near future, and things look good for that. The most common solution, AFAIK, is the GLOBUS grid middleware - it standardizes a lot of this stuff. It is imperfect and needs work - but it's coming along. Previous comments that Globus' UI is imperfect are silly - it's like carping about the UI of a machine-code instruction. Others middlewares also exist in some form or other, and a few will eventually emerge as solutions to the various problems. Again, the solution you use will vary according to the problem you face.
Eventually, the goal is to have a fairly portable, secure, flexible framework that will run a reasonable number of applications on a reasonable number of platforms. The software has to be admin-friendly - no need for root on the boxes, easy to set up, easy to remove. These are all adjustable requirements.
Right now I run massive high-energy physics software (preinstalled on the site) on tens of sites in the US and Korea. It's not user-friendly - it's in development. It's extremely powerful, even now - my jobs are gargantuan and the disk to store them even more so. I need only one application set, and I have it. In the future, things will be easier - this is all in development mode. However - it DOES work, it WILL continue to improve, it continues to become more secure, and someone (when things are ready) will code up an interface to it that will make it friendly.
That's pretty good. Naysayers take note. This isn't a vaporware idea - it's just a difficult problem with a lot of blanks left to fill in.
DNA with M&M's. You'd have to have a planet's worth of M&M's to do anything useful. Seriously - I kept watching because I was sure it would eventually get better. Right?.......
Never. Fantastic movie. Loved the pasa double. Hilariously funny. Of course, you have to have a certain sense of humor. Cold Comfort Farm and O Brother Where Art Thou fall roughly in the same bin.
That being said - just sit where you can see best. Same as I do watching movies on my laptop.
The projector (both in stills and on the avaliable film) compensates for both surface color and configuration automatically and applies the transformation on a pixel-by-pixel basis. Good idea, IMHO. Sure seems to work. Seems to be arbitrarily flexible, to first order.
There are definitely artifacts, but they seem to follow curved edges - not surprising when one realizes that pixels are finite and rectangular.
I wonder how the reflectivity of the surface affects the adaptation.
Watch the movie - it's pretty clear on the process.
Best software I ever tried for reading is Weasel Reader, hands down. Mentioned further down.. look into it. zLib compression halves the size of your book, and on-the-fly decompression is very fast. And it's open source. It also lets you rotate your text into landscape or reverse-landscape - very convenient reading format.
Files are generated from text using a small linux program called makeztxt. I've built scripts around it to clean up, edit and compress all of my text files, and keep them organized. All of this is doable on OS X as well.
Get a memory card. Yeah, I know it's not cheap. Get one that matches your digital camera so it can serve double duty. Put all your books on that - you'll hate having to sync new books on (slow) and having to go back to your computer to get another one when you finish. Especially if you accidentally hard-reset. At the airport. :) Keep your whole library with you. Have a selection.
Don't get a non-backlit screen unless you have to. Color isn't necessary, but non-backlit screens are very hard to read in twilight conditions, even when you turn on the 'glowing text'-type lighting.
Resolution doesn't matter. You can read the letters just fine in 160x160, and the higher-res screens _seem_ to have less real estate - thus, smaller letters. The backlit screens are definitely smaller. 'Course, it's easier to see a small letter on a luminescent background than a larger letter on a gray background. Then again, backlights chew battery mercilessly.
No kidding. Especially since it sounds more like a renderfarm than a single "supercomputer". Then there's the data storage. That's nothing (though I say it myself). Go to high-energy physics for serious storage/processing centers. Little ol' me uses ~15 TB from 4 months of work, and I'm nothing compared to some people's requirements. I use up about 0.4% of our mass storage here (rough guess). WETA's cool - but not in supercomputer/HPC land.
ftsh has great utility in the realm it's written for. Obviously, it's not a basis for installing kernels or doing password authentication. In a Grid (not just distributed) environment, things break for all sorts of reasons all the time. You're dealing with a Friendly Admin on another system, one who may well be unaffiliated with your institution, project or field of study. He doesn't have any particular reason to consult with you about system changes.
Now you find yourself writing a grid diagnostic or submitter or job manager. One does not need strongly typed compiled languages for this. Shell scripts are almost always more efficient to write, and the speed difference is unimportant. Right now, most Grid submitters are being written in bash or Python or some such. Bash sucks for exception handling of the sort we're talking about. Python does better with its try: statements, but there's room for improvement. ftsh is a good choice for a sublayer to these scripts. One writes some of the machinery that actually interacts with the Grid nodes and supervisors in this easy, clear and flexible form.
Now there are a lot of specific points to answer:
One needs a Windows port to be able to make the Grid software we write in Linux available to the poor drones who are stuck with Win boxes.
This is not a code spellchecker or coding environment. At all.
This is not a crutch for inadequate programmers. This is a collection of methods to deal with a specific set of recalcitrant problems.
As I was pointing out before, this is, after all, an unstable system. One is using diverse resources on diverse platforms in many countries at many institutions. I appreciate the comment made by unixbob about operating in heterogeneous environments.
This isn't a substitute for wget. One uses wget as an example because it's clear.
The "pull" model breaks down immediately when there is no unified environment, as is described on infrastructures.org. When you're not the admin, and your software has to be wiped out the minute your job is done, "push" is the only way to do it. This is the case with most Grid computing right now (that I know about)
:) Use the right tool for the job and save time.
:)
All the woe and doom about the sloppy coding and letting the environment correct your deficiencies is... ill-thought-out. That's what a compiler is, folks. Should we all be coding in machine language?
I do agree, however, that one should indeed hone one's craft. Sloppy coding in projects of importance is inexcusable (M$). There is no reason to stick to strict exception handling, however, in the applications being discussed by ftsh's developers (the same folks who brought you Condor). When code becomes 3/4 exception handling, even when the specific exceptions don't matter, there's a problem, IMHO.
Here's a recent overview article on the status of Higgs in the LEP data (refinement and rehashing of stuff that's not really new anymore). Go to http://arxiv.org/PS_cache/hep-ph/pdf/0402/0402231. pdf
The total LEP experiment sigma comes out as less than two for a 115 GeV SM Higgs. That's not compelling. However, some VERY nice "gold-plated" 4-jet events were seen in the ALEPH detector, and it seems like there's a good chance that 115 GeV will be a good place to look in LHC.
Speaking of LHC, here's a webcam that lets you look at the ATLAS detector being built. :)
http://atlaseye-webpub.web.cern.ch/atlaseye-webpub /web-sites/pages/UX15_webcams.htm
Hm. Go learn some of the math involved, and come back when you understand that there really are some compelling reasons for Higgs to come into the picture. Or are you aware of something that's been overlooked? You have a good reason that photons are massless and W/Z bosons aren't? Can you tell me why electrons weigh less than taus? Can you tell me how "mass" comes about? That plus the fact that the possibilities include standard model Higgs and SUSY Higgs, light Higgs, heavy Higgs, MSSM doublet Higgs, all in different variations... We didn't ask for all these particles to show up. We're just trying to figure out what we're seeing in nature.