Slashdot Mirror
Cool/Weird Stuff To Do On a Cluster?
Gori writes "I'm a researcher at a university. Our group mainly does Agent Based Modeling of interdisciplinary problems (think massive simulations where technology, policy, and economics meet). Recently, we managed to get a bunch of money for a High Performance Cluster to run our stuff on. The code is mostly written in Java. Our IT support people are very capable of setting up a stable cluster that will run Java perfectly. But where's the fun in that? What I'm trying to figure out are other, more far-out and interesting things to do with this machine — think 500+ Opteron cores, 2 GB RAM per core, a gigabit interconnect with some badass switches, a massive storage array, plus a bunch of UltraSPARC boxes. So at times when there's no stuff to crunch, I'd like to boot the thing up with a 'weird' system image and geek around in the name of science. Try fancy ways of building models, dynamically adding all sorts of hardware to it, etc. Have different schedulers compete for resources. Imagine a Matlab vs. Boinc vs. ProActive shootout. Maybe run plan9 on it? Most of us are not CE/CS people, but we are geeky enough. So, what would be the coolest and most far out thing you would do with this kind of hardware ?"
And save the environment a little bit?
"So, what would be the coolest and most far out thing you would do with this kind of hardware?"
Instead of pissing around with stuff that may not go anywhere other than a few giggles over lunch.
Why not just rent, or lend it out to people who don't have the funding or equipment that could use this cluster for a better purpose than "playing around"?
Just saying...
I (Mat) work at Vanderbilt University's supercomputing center. Our university supercomputing center was originally a joint venture between the proteomics and high-energy physics departments, but they decided to make it a independent university-wide initiative to bring HPC tools to all users.
Before we founded the center, there were a lot of groups that required computing on campus, but it was highly inefficient. Their local clusters had lots of free cycles (low return on asset) that they couldn't effectively share with other users, the clusters were down quite often (grad students and postdocs are poor sysadmins, plus they should be doing actual research anyway). Several other problems related to either pooling of resources or pooling of knowledge, you get the idea...
I highly recommend setting up a batch scheduler such as Torque/Maui and opening your cluster to all researchers on campus. You'd be surprised how much demand is out there. We have all the usual math/science/engineering/biomedical users, plus users in more esoteric fields (nursing, accounting, music, psychology). You can always give your group a higher job priority if needed. It gives a higher return on asset and gets lots of goodwill on campus (and, potentially, at funding agencies). You can charge users for support, storage, etc for cost recovery, or even use it as a revenue source if your grants allow.
Having different types of users also allows cross-pollination of ideas. We have a large number of biomedical researchers who are now using a high-energy physics software (geant), biomedial people who are teaching other users how to program in R, etc. These are avenues for research/discovery that didn't exist before.
Sig? What's that? Oh, 'signature'...and it's supposed to be witty? Right...
You should maybe hide your signature when you say things like that.
Itg is widely accepted that Vista is a waste of resources, and therefore all articles about powerful computer resources are going to have Vista jokes. You can't kill off a meme on your own. If you don't like repetetive humour then perhaps you should change your moderation to mod down all funny comments.
Anyway, slightly back on topic - I think they'd want to keep it a cluster rather than degenerate into a clusterfuck, so it may be better to avoid installing Vista.
which is totally what she said
In my experience as a sysadmin, when you have a resource, your users want it to be "up" all the time, no matter what. If it's interactive, they'll leave VNC sessions or xterms or screen sessions running on it and want them to be there when they come back. If it's noninteractive (ie: a queue/batch system), your users want to be able to submit jobs now, without waiting for the sysadmin to come in and fire it all up and make it run.
Without some serious organizational political capital, it's pretty hard to pull off powering down the compute resources. It can be done, but it's going to leave a lot of people unhappy.
Uh. So the endless Beowulf jokes are funny to you, but the nowhere near as long running Vista ones aren't? You perhaps should go into therapy to resolve your Vista issues. I'm intrigued as to why you consider it better than XP in any way.. apart from apparently the calendar is more comprehensive than XP for tracking changes in our date system over the last while.. but other than that. I honestly don't see the benefits. I'm not being hypocritical either - I was doing fine with 98 until games started requiring XP. And I'm writing this on OS X.
which is totally what she said
Well, you could always make a virus farm
But seriously though, let your physics department know that you have idle time on such a cluster, they will come begging at you for some cicles. I've tried running a simulation of some quantum systems from first principles, using Monte Carlo methods (my software), and it would never get anywhere on small clusters. On a big one I managed to see something, but after days of processing.
entropy happens
Couldn't you be doing something yourself to save the environment right now - something like turning off YOUR computer instead of browsing slashdot? Oh, wait - that would require YOU to do something. That kind of environmental activism is never as much fun as simply preaching to other people what THEY should be doing. Hypocrisy is SO much fun - carry on!
Bullshit. No way, no how is C++ ten times faster than Java. You're smoking crack.
ZFS: because love is never having to say fsck
Huh? This kind of clueless drivel marked as "insightful"???
It is doubtful that there is significant amount of overhead. Fortran could be faster for specialized math tasks, for few select cases maybe an order of magnitude faster. C++ would just crash more often, without guaranteeing ANY speedup. Competently written java code, for agent-based simulations, should be up there with compiled languages.
Hacking reality? That sound like Engineering to me.
I think reverse-engineering reality is called "Physics".
I thought you were going to suggest that he copyright all of the 255x255px bitmaps that weren't already copyrighted and then use the machine to look for offenders and mass e-mail take down notices to them.
C.
Also, 20% is no small number. (Damn filter. Ruined my one-character post.)
I just read Slashdot for the articles.
nobody wins that last one...
Obama is a twitter sock puppet
Why, what's wrong with using Java? Really I'm not being sarcastic I want to know your reasons.
http://en.wikipedia.org/wiki/Comparison_of_Java_and_C%2B%2B#Performance
You would also have everything possible within 255X255, but as images can be tiled, you would also have an almost infinite canvas with infinite possibilities.You want a time machine? Well, if it's possible, then SOMEWHERE in these images is the blueprints, chopped into nice 255X255 chunks.
As well as all porn being thumbnailed, why not find the tiles for it in HD resolution instead? Or better yet, in 10000X10000 pixels?
So in essence, such a program would spit out every frame of every movie, in better than current HD resolution, that will ever be made by humans (or aliens, or anything). It would also contain the entire set of human books, past and future, and alien books for that matter, along with the correct method to translate it. It would have every bit of knowledge possible to fit into a 2 dimensional representation written down for us to read right now.
Which is all very exciting until you realise that it is, of course, not possible to do with computers in any meaningful timescale within a Universe, and for every correct set of blueprints for a time machine there must be an almost infinite set of incorrect ones which are indistinguishable from all the others until you build it.
As an interesting aside, I wonder what size bitmap it IS capable for - 2X2? 4X4?
Technically, this is overkill. You get the exact same thing by running through all the possible colors for a single pixel. All you have to do is combine those pixels in an infinite number of combinations and you have everything in the world. Extrapolating to 255x255 tiles first doesn't get you anything that you don't have with the original pixels, other than a storage and sorting problem.
The Quirkz Handbook of Self-Improvement for People Who Are Already Pretty Okay