Slashdot Mirror
Cool/Weird Stuff To Do On a Cluster?
Gori writes "I'm a researcher at a university. Our group mainly does Agent Based Modeling of interdisciplinary problems (think massive simulations where technology, policy, and economics meet). Recently, we managed to get a bunch of money for a High Performance Cluster to run our stuff on. The code is mostly written in Java. Our IT support people are very capable of setting up a stable cluster that will run Java perfectly. But where's the fun in that? What I'm trying to figure out are other, more far-out and interesting things to do with this machine — think 500+ Opteron cores, 2 GB RAM per core, a gigabit interconnect with some badass switches, a massive storage array, plus a bunch of UltraSPARC boxes. So at times when there's no stuff to crunch, I'd like to boot the thing up with a 'weird' system image and geek around in the name of science. Try fancy ways of building models, dynamically adding all sorts of hardware to it, etc. Have different schedulers compete for resources. Imagine a Matlab vs. Boinc vs. ProActive shootout. Maybe run plan9 on it? Most of us are not CE/CS people, but we are geeky enough. So, what would be the coolest and most far out thing you would do with this kind of hardware ?"
And save the environment a little bit?
I'd be interested to see how quickly you could fold@home@work on that cluster.
I think that would be an ideal setup to run Vista on. I heard that with SP1 you might only need half of the equipment you currently have.
that it will be fast enough to run Duke Nukem Forever when it gets released.
"Prefiero morir de pie que vivir siempre arrodillado!"
a beowulf cluster of these .. .
Humor from a Genetically Molested Mind
"So, what would be the coolest and most far out thing you would do with this kind of hardware?"
Instead of pissing around with stuff that may not go anywhere other than a few giggles over lunch.
Why not just rent, or lend it out to people who don't have the funding or equipment that could use this cluster for a better purpose than "playing around"?
Just saying...
Because Gentoo is just hours of fun....
compiling.
:-)
power it off and go get a blow job. You might like it.
You have a high-performance cluster, and you're running Java? I bet the OS in windows, too.
Have an open framework for members of the open source community to enhance and create new cluster/multicore optimized technologies. Allow us to do render farms or computation of nodes to do whatever sounds cool at the time. Of course the projects have to be approved, and their may be a small fee unless the project looks interesting or you wish to claim it against a donation of resources.
Get it to play a game of Tic-tac-Toe against itself. Give us some WOPR love!
"The price good men pay for indifference to public affairs is to be ruled by evil men." ~Plato (427-347 BC)
Being massively parallel it could be interesting to do some AI research. Like chess algorithms or something more fun perhaps a good automated tech support system via chat. Or trying to decode, capatacha. Map optimizations... a bunch of fun stuff. At least it would be for me.
If something is so important that you feel the need to post it on the internet... It probably isn't that important.
You know you want to. Or you could evolve a bunch of virtual creatures on it: http://www.karlsims.com/evolved-virtual-creatures.html http://www.stellaralchemy.com/lee/vce3d_related_projects.html
Brute force random sizes of pe code or image code based off of known patterns or headers or signatures and see what interesting new things can be created.
Instead of random, how about have algorithm of the hour and show what seeding or using it can do.
Chess
Poker
Fighter Combat
Guerrilla Engagement
Desert Warefare
Air-to-Ground Actions
Theaterwide Tactical Warfare
Theaterwide Biotoxic and Chemical Warfare
Global Thermonuclear War
??
Write a program that creates every possible 255x255px bitmap possible, and then right an algorithm to go through and figure out which ones are rubbish and which ones are actually recognizable pictures.
Logic dictates that one of the resulting pictures would have to be of John Lennon kicking George W Bush in the nuts, find that picture and post it on failblog.
Voila.
Protein folding really helps us biologists A LOT, so please run it. http://folding.stanford.edu/
There was this one incident when me and this girl turned a cabinet over on it's side and used it as a surface for sex play. That was different. It didn't get weird until she insisted that I tie her wrists with some Cat 5 though.
I'm doing computational condensed matter physics for my PhD. If you give me an account, I can keep your cluster at full load while you're not using it. :) The type of stuff I run can span hundreds of processors over (sorry to say it) slow interconnects like GigE.
If you let me use it, I'll acknowledge you in all publications AND I'll make you some pretty pictures (materials under extreme conditions make for cool figures).
Anyone else think a 2^16 vs. 2^16 Team Fortress game would be pretty cool?
Corollary to Hanlon's razor: Any significantly advanced stupidity is indistinguishable from malice.
as some have said, donate the time to people would like to buy it but can't, or on odd days turn up 1000 SETI@home processes and see how far it gets?
Create a reverse Google bomb - Index every link to say 'george bush' from Google, read each page into memory hash the words, assigning value by count amongst all the pages, and then post the top ten words on GW that are not 'the' 'a' 'was' etc.
Perhaps comparing this to the same process on Paris Hilton would be a report that sparks SkyNet to life. I don't know. Seriously, if those two have 6 of the top ten words in common it would have to mean something.
If you have all that cpu sitting idle, that is the kind of weird shit nobody else would do, but also couldn't not read the report either. Perhaps you'll start something new for Google Trends? :)
Support NYCountryLawyer RIAA vs People
I do burn-in testing on a lot of the machines we get at work, using hardware I can't personally afford.
My favorite test is to find scenes to render with a raytracer.. yafray is my favorite, runs on all major platforms. But not just any scene, it has to have all the details turned up to 11, contain extremely high detail (polygon counts drive up memory usage), and write out an absurdly large image.
Kind of whimsical but it's hard to not be impressed by an image 20,000 pixels square with perfectly accurate reflections. Who cares if I can only fit a fraction of it on my monitor. ;)
Rosetta @ home or fold.it
Or you could try to thermally load them in patterns that produced different tones on the fans (or maybe an AM radio) record it then speed it up. Maybe you could make it sound like a baluga whale.
Or maybe you could implement a virtual machine cluster of 250 cores. The repeat the process till you see how many virtual machines you can stack on top of each other till it has the same speed as a single processor.
While this might sound stupid this would give us a rough estimate of how many watts per virtual world it takes and from that we could figure out how many layers deep in the simulation we actually live assuming the top level one is powered by something less than 1 sol of power.
or work out all possible moves in N-space tic-tac-toe. The only smart move is not to play.
Some drink at the fountain of knowledge. Others just gargle.
Play the original Tetris game one the most powerful computer it's ever been played on. That's the first thing I would do.
"So, what would be the coolest and most far out thing you would do with this kind of hardware ?"
Get Rails to scale?
I (Mat) work at Vanderbilt University's supercomputing center. Our university supercomputing center was originally a joint venture between the proteomics and high-energy physics departments, but they decided to make it a independent university-wide initiative to bring HPC tools to all users.
Before we founded the center, there were a lot of groups that required computing on campus, but it was highly inefficient. Their local clusters had lots of free cycles (low return on asset) that they couldn't effectively share with other users, the clusters were down quite often (grad students and postdocs are poor sysadmins, plus they should be doing actual research anyway). Several other problems related to either pooling of resources or pooling of knowledge, you get the idea...
I highly recommend setting up a batch scheduler such as Torque/Maui and opening your cluster to all researchers on campus. You'd be surprised how much demand is out there. We have all the usual math/science/engineering/biomedical users, plus users in more esoteric fields (nursing, accounting, music, psychology). You can always give your group a higher job priority if needed. It gives a higher return on asset and gets lots of goodwill on campus (and, potentially, at funding agencies). You can charge users for support, storage, etc for cost recovery, or even use it as a revenue source if your grants allow.
Having different types of users also allows cross-pollination of ideas. We have a large number of biomedical researchers who are now using a high-energy physics software (geant), biomedial people who are teaching other users how to program in R, etc. These are avenues for research/discovery that didn't exist before.
Play a digital version of hide and seek. Flip a certain register on only one of the memory chips and have competitions on who can find it the first.
See what it takes to make the processors burn.
What to do with it? Thats easy. Build a hot chick like they did in Weird Science then GPL the code.
Doooo iiiiiiiit!
Try figuring out a new prime number, perhaps win some money too mersenne.org if you can figure out the first 10 Million Digit Prime.
http://www.opensciencegrid.org/
That way you can backfill the resource with jobs from VOs (Virtual Organizations, think projects/groups) you choose. It is similar to BOINC in the sense you can pick what science to support.
The idea is that the sharing will go both ways. You will give spare cycles to other users on OSG, and in return, you will be able to use spare cycles on other resources.
Spend some time on the OSG website reading the things under "Learn About Us". Also check out the research highlights to see what kind of science is being done on OSG resources:
http://www.opensciencegrid.org/About/Research_Using_OSG/Research_Highlights/Research_Highlights_Archive
(Shameless plug - I'm part of this team) The good news is that OSG have an ongoing effort to help you join your resource to OSG, and help your users get going on the grid:
https://twiki.grid.iu.edu/twiki/bin/view/Engagement/WebHome
xkcd Network
Instead of randomly generating pictures or something as someone suggested above, why not make it generate say a 64k program, keep iterating through this until you get an executable that will actually RUN (Without crashing) and see what happens.
+1 IDisagreeSoHeMustBeATrollOrAnAstroturferOrAShill
Would be to run a VR simulation of you interacting with female simulacra in social situations while simultaneously running a parallel statistical model of the chances of you getting laid based on your interactions with the simulacra.
Sig this!
In my experience as a sysadmin, when you have a resource, your users want it to be "up" all the time, no matter what. If it's interactive, they'll leave VNC sessions or xterms or screen sessions running on it and want them to be there when they come back. If it's noninteractive (ie: a queue/batch system), your users want to be able to submit jobs now, without waiting for the sysadmin to come in and fire it all up and make it run.
Without some serious organizational political capital, it's pretty hard to pull off powering down the compute resources. It can be done, but it's going to leave a lot of people unhappy.
How about simulating tentacles physics for future 3D hentai games?
Prize money in the $100K and up ranges are available for primes that are found above 10 million digits. If you can get the cluster to search for a 10 million digit prime and you end up finding it, you'd earn your Department a cool $100,000 in extra funding. I haven't read the exact rules for the Prime95 program invlovement, but you'd still get a good grant for your Dept. from just crunching 1s and 0s on your idle processes.
I'm a CS (Anthro minor) undergrad student working with "Agent Based Modeling of interdisciplinary problems" looking for a place to go to grad school. What university is this with the nice cluster and the group of people working on the problems I want to help solve?
Well, you could always make a virus farm
But seriously though, let your physics department know that you have idle time on such a cluster, they will come begging at you for some cicles. I've tried running a simulation of some quantum systems from first principles, using Monte Carlo methods (my software), and it would never get anywhere on small clusters. On a big one I managed to see something, but after days of processing.
entropy happens
Couldn't you be doing something yourself to save the environment right now - something like turning off YOUR computer instead of browsing slashdot? Oh, wait - that would require YOU to do something. That kind of environmental activism is never as much fun as simply preaching to other people what THEY should be doing. Hypocrisy is SO much fun - carry on!
A variant on the rendering theme: Instead of simulating rays, simulate light wavelets and the modern theory of light. You should have enough compute power there to render reasonable scenes using such a technique.
Fluid dynamics: This is a popular one, and NASA offer source codes for free for subsonic, supersonic and even hypersonic flows. In fact, they offer quite a number of subsonic ones. They're also not the only source. There's several open-source CFD packages, ranging from river simulations to aircraft simulations.
Supernovae simulation: There are packages (freeish, rather restricted in access) that allow you to simulate thermonuclear and supernovae explosions within stars. The restrictions are for rather obvious reasons, even though the odds of anyone nasty obtaining a star is, well, unlikely.
COLOSSUS: There are still a couple of ENIGMA ciphers that have never been broken, which can be obtained along with the algorithm Colossus used in World War II to crack such codes. You could complete the set and maybe discover some lost secret (yeah, right).
BLAST: Other posters have suggested renting out the computer time, but that just transfers the problem of what to run, rather than solving it. BLAST, or one of the MPI-based variants thereof, is an exceedingly popular tool for examining nucleotide sequences, but as the databases grow ever-larger, the demand for ever-more information also increases, creating a need for significant compute power to produce the volume of results desired.
It's a small world and it smells funny; I'd buy another if it wasn't for the money; Take back what I paid (SoM)
Bullshit. No way, no how is C++ ten times faster than Java. You're smoking crack.
ZFS: because love is never having to say fsck
Obviously they need to upgrade their equipment.
That actually does sound interesting!
Even with one node on all the time... you could pretty reasonably pull that off with the rest of the nodes. You could even have it dynamically scale the "awake" part of the cluster to match the number of jobs in the queue or the utilization of the awake parts. Just imagining going from 200W to 30-40kW and back automatically, on demand... yeah, that'd be handy.
You could also tie into environmental factors ... scaling back the cluster if the HVAC is underperforming and the room's getting warm. Or tying in power and cooling load balancing, powering up appropriate systems to keep the room from getting hot and cold spots.
Huh? This kind of clueless drivel marked as "insightful"???
It is doubtful that there is significant amount of overhead. Fortran could be faster for specialized math tasks, for few select cases maybe an order of magnitude faster. C++ would just crash more often, without guaranteeing ANY speedup. Competently written java code, for agent-based simulations, should be up there with compiled languages.
If an infinite number of monkeys......
Write a program that creates random code and tries to run it, then starts again. Eventually, we'll have the Singularity and not have to worry about finding cool things to do.
Stasis is death. Embrace change.
You must be new here (I guess this one also qualifies as a classic)
I think it is...
under construction
I've worked with supercomputer class systems in the past, common question - "what can I do now?"
Two suggestions:
1. Set aside the first couple "downtime" sessions to run your crazy "OMG it actually worked" stuff out of the way. The stuff you'll brag about for years to come.
2. Every time thereafter, set it up with a grid computing client. Seriously. It may not take advantage of the interconnect fabric, but you will be doing your fellow humans a great service.
Check out http://www.worldcommunitygrid.org/ for some ideas (from a big company whose sponsored code will not cause infinite loops on the nodes....).
SETI is IMNHO a complete waste of time.. if or when the aliens want to be found, it will probably be in Hollywood ID4 style and not some little green guy screetching "Can you hear me now?" on all frequencies and modulations.
Doesn't using Java nullify/negate the power/usefulness of the cluster?
from 09 F9 11 02 9D 74 E3 5B D8 41 56 C5 63 56 88 C0
to 45 2F 6E 40 3C DF 10 71 4E 41 DF AA 25 7D 31 3F
Reading Slashdot dulls creativity sometimes... I suggest you deprive yourself of sleep, overdose on caffeine, read something by Neal Stephenson (Snow Crash or Diamond Age), and meditate on your question with a 2-3 geeky buddies of yours around 2am.
1) Find a picture/piece of code/ISO image/etc. you'd like to compress. Treat it as a very big number. Now find a prime number that comes very close to it. Compress it by treating it as the Nth prime + the remainder. Repeat for the remainder See if the compression is any better than bzip. (I think i saw something like this done for the DeCSS code once...)
2) Find something to optimize via brute-force. (My favorite is neural nets predicting time-series). Run a distributed simulated annealing algorithm. (Run an instance of the algorithm on every core, check every N cycles to see which core is more "optimal", share the parameters with the rest of the instances).
3) Create an interactive multi-user raytraced environment. Get someone to bring in an ImmersaDesk or a CAVE. Stun all visitors.
4) Model a giant neural network. Teach it to do something cool in real-time with your favorite training algorithm...
5) Get dictionaries from 50-60 languages. Write something to correlate similarity of words meaning the same thing. Make pretty graph clustering them by similarity. Post pretty graph on Slashdot.
6) ???
7) Profit!
Because the Java codes I have to optimise to run on our cluster (or SMP machine) still run like lame, wet dogs. I'm sorry, but Java's performance is really, really shitty and I'm considering removing Java and Javac from the systems altogether. When you compare another object oriented language, like C++, at least with that you can write your codes to make use of multiple instructions per cycle ... and then with a good compiler and appropriate hardware, you can vectorize the instructions themselves. When you have codes that for a single processor, would run for about 50.9 years (an example I pulled from just today) those 'simple' little things go a long way. Now where can you tweak that sort of thing in Java? ... Oh, that's right, YOU CAN'T.
But that's object oriented languages (which is getting better since I'm learning how to make myArray[i] run twice as fast), I prefer to use C and fortran. Those babies are low memory and totally rock. You don't spend all your time making method calls, which chew cycles for each call (let alone being in large nested loops).
I don't mind punching out Java code; it's extremely good for some tasks, but it is not a language I'd use on a high performance system. Just in my experience. I could be wrong and would love to be proven wrong.
.
Here's some speculative science that might be worthy of modeling: Ionic Kinetic Energy Conversion Effect
Are there limits to the fact that when a charged particle is accelerated, it emits a photon? I was once told that below a certain point, the charge does not emit a photon. Really? Why? A possibly useful phenomenon needs relevant data! Thanks!
A number of other posters have suggested this answer, but I wanted to emphasize it.
You are lucky enough to have a massive, desirable resource that is underutilized while we are in the midst of an academic funding environment which has been called the worst ever for supporting basic and applied research. You, as an academician, have an unwritten duty to help other academicians, and so rather than think of geeky ways to mess around with your new toy, why not share your wealth?
Here are the advantages:
1. You can wrangle additional publications out of it by being an author on published papers by providing sufficient support. Publications are the lifeblood of academics.
2. You can use it to leverage improved relations (or establish relations) with other departments within your university, or across universities. This might not seem like it would be worth much to you, but it will be impressive to your supervisors and department heads (they can take credit for it), and will make you look good in their eyes.
3. You can leverage it to good-news PR. University administrations love good PR. Talk to your PR department. Involve your department.
4. You can do good in the world.
I, personally, run an analysis of neural recordings every so often using Matlab that takes about a week on a reasonably modern dual-core system. It sure would be nice to have that finish in 1/500 as much time! There are about 20 or so researchers who are doing work like mine, and none of them, to my knowledge, are using a high-performance computing environment to analyze their data. It just isn't within their means. I'm not talking about some esoteric, arcane basic science research (although I'm a huge, huge proponent of that), but helping the paralyzed to walk again.
Put my fist through my alarm clock with its ding-dong death inside my ear. - The Blackjacks.
And save the environment a little bit?
:P
Try to find the least power consuming configuration that works under full load.
Then, starting from that point try to find the best way to automatically adjust between idle and full load (think disabling/enabling cores).
After you're done compare the power consumption with that after your modifications, translate that into management speak ($/year saved) and go get your fat bonus.
Oh, once you get it you owe me a beer