Slashdot Mirror
The Truth About SETI@Home
zealot writes "According to this article, the SETI@Home project is not using the most optimized clients available "just to brake the unit turn around" so that they can continue to recieve various contributions. The authors are also demanding access to the client source (and asking to GPL it if possible), so the greatest performance may be obtained. " It's an interesting point: They didn't figure on getting the reponse they did, and will sooner rather then later run out of blocks to be crunched. Yep: What happens if hold a war and /everyone/ comes? Or a distributed program, I guess.
I would rather see that they not GPL the client. It would be far too valuable under a BSD license or an X Consortium license. Whereas if the software were placed under the GPL the code becomes useless for general use.
All of the evidence points to exactly the problem that the author describes. Seti@home needs to write a second client to handle the division of work and "graduate" some of the reliable, high performing volunteers into producing the units of work for others.
Their fundamental problem is that they only distributed the analysis portion. Now that the overall load has become unbalanced, they need to distribute one more piece of the workload.
Geeky modern art T-shirts
Opening up the source code to the world will increase the block counts in two ways:
1) Optimisation
2) Hacked clients returning blocks unchecked, as previously seen with distributed.net
By using a hacked client and _lots_ of different
e-mail addresses to report completed blocks with, any hacked-client-antics could go undetected....
I'm not saying that this is unbelievable, just that it would be nice to have some evidence to back these claims up... or else state them as conjecture, not fact.
--
Let's not get too wound up because SETI@HOME is getting overwhelmed. It means that the idea is successful. Why do people participate in these kind of distributed processing projects? With the exception of those who want to show off their machine, most of us do it because we feel that instead of our computer downloading Warez or porn all night, we can do something useful.
If SETI@HOME is having some troubles, helpful advice, not scathing criticism is what is needed.
I would much rather see more of this kind of thing, even if it was occasionally bungled, than other groups being scared off because of how hostile the online community is
Maybe I'll find ET...
-------------------------------END--COMMUNICATION
The article in 3DNOW is way off base... If you check the seti@home home page (http://www.setiathome.ssl.berkeley.edu) they explain why the code is not released to the public. Additionally, the code improvements they discuss are limited to Intel chips, and I, for one, am not running the bulk of my setiathome clients on Intel chips (too slow...), although I realize that a good portion of the clients being run are Windoze clients.
From the Yahoo page:
Looks like anyone interested can find out the real scoop from the horses mouth.
The article seemed to be flame bait to me. They never said that Seti@home said anything other detailing the performance critical routine in the seti@home software. Then the way I read it seti@home did not want to give up their source. The article said:
Is this what they said or more likely an interpertation of what they said?
Lets check the facts before slamming Seti@Home.
Check out the Lance Armstrong Foundation
kayaking
(Joke! :))
www.HearMySoulSpeak.com
They're worried about running out of blocks to process? Isn't that the idea?
The problem is that the data they want to process doesn't come in blocks, it comes in one big chunk (or several), and they can't break it up fast enough to keep up with the blocks being finished.
---
END OF LINE
I've been recieving ancient Seti@home units for weeks. They keep sending out the same units over and over, from January 1999 through March 1999. When you work on it, you notice that it just keeps looping. You hit a February unit, then a March unit, then a January unit, etc.
:)
Maybe they've set it up to analyse the units in that strange order, but I doubt it seeing as they have enough computing power to encode several thousand years of MP3's per day... using BLADEENC!
What the Seti@home project needs is a way to better manage the data that they're being sent. It would also be nice if they'd optimize their client so we could run it for the same amount of time, but have it take up less CPU. It really is intrusive when it's not running in screensaver-only mode under windows, and without -nice 19 in Unix.
High Energy Physics experiments use vast quantities of computer time to analyze results from Terabytes (now, soon Petabytes) of data and even vaster quantities in Monte Carlo programs to generate events to under detector biases and data backgrounds. In both cases, data is discrete events. Fermilab was one of the pioneers of production farm computing - a loosely coupled multiprocessor system. The canonical farm consists of a horde of low-cost systems (workstations or PCs running Linux) that are the worker nodes plus a small number of systems which act as overall managers of the process and which parcel out the data to worker nodes. In the past the I/O nodes read the input tapes, gave a single event to each worker node, collected the results and wrote the output tape(s). Modern implementations have the worker nodes acting on a file containing 1 to a few thousand events.
I've wondered if the Monte Carlo processing (in particular) would be ameanable to SETI@home-like distributed processing. One possible problem is that the analysis and Monte Carlo generation programs are often quite large and have largish memory requirements.
Dr. Frank J. Nagy Fermilab Computing Division Authentication and Directory Services Group
Oh damn, the Seti@HOME people are "making" me run my computer all night (at "full throtle", no less !
My opinion of his opinion: he should "get over it". The only thing driving that dude is competition with others, not the altruistic donation of *spare* computing power towards (an arguably) good cause.
If I ever write an article like that, remind me to switch to decaffinated coke.
-adam a
Am I the only one sick of seeing this argument used against SETI@Home? This is totally STUPID. My computers would run the same hours with or without SETI@Home on them. I also don't think that I'm pulling *that* much electricity to make a large impact on the atmosphere - and my area uses hydroelectric power anywayz.
I think it's funny though when people hear this argument and then use it to advocate d.net.
Err, distributed.net is not open. The fact that the distributed.net client was hacked demonstrates, once again, that security through obscurity does not work. In other words, trying to make it more difficult to hack the client by only publishing the object code rather than the source code as well is a much worse solution than figuring out some way to guarantee that bad blocks cannot be submitted, regardless of the functionality of the client.
I like to think of distributed net in the same context as the NASA Deep Space 1 mission that just flew by comet Braille.
-DN (distributed.net) has had it's share of glitches and trouble, but now it represents a technology that we can apply to other problems.
-DS1 has had a few glitches of its own. The ION engine wouldn't start properly, and then it mysteriously started working after a while. Of course, we figured out what was wrong, and that seems to be a normal characteristic of brand new ION engines. Overall, the DS1 ION engine has operated for 1800 hours, vindicating the original concept.
-DN tackles a current political issue, but the problem is technically boring. Cryptology is hot in the news today, but we all know the outcome of the DN problem. We'll find the key. But along the way we will learn a great deal about how to build vastly distributed programs running with donated computer time.
-DS1 also tackeled a current political issue, but it was essentially boring. DS1 flew by astroid Braille. Asteroids are in the news, but DS1 was so far away from it that the asteroid occupied only 4 pixels in the CCD. It also appears that there was a problem with the tracking system, so there might not be any better photos. Boring! But we're learning a tremendous amount about how to build spacecraft that can automatically perform their own navigation.
SETI at Home's biggest mistake is that they re-invented the wheel and made all the mistakes they would have avoided if they'd had some help. They have the right problem. We're all interested in finding alien life. But they could learned something from the distributed net people.
If tits were wings it'd be flying around.
Look at the awful job most of the search engines are doing keeping up with the web. Why not a distributed spidering project? Hand out a base of URLs to spider, then let remotes spider from there. As the ever pessimistic Rob has already pointed out to me, the load on the host end would be huge, but I still think if it were done right, the whole net could be cataloged in a few months, then kept updated.
It seems like a distributed spidering project with a search engine front end like Google on different hardware/net could make a search engine useful again. There are probably a few interesting things that sites could do to streamline the workload--I haven't thought of them, but my spidie senses are tingling.
slashdot broke my sig
Check out COSM, which is supposed to do exactly the sort of thing you describe.
It's a small world and it smells funny; I'd buy another if it wasn't for the money; Take back what I paid (SoM)
Seems to me the simple answer is to process all blocks twice and compare the results. This would then solve/detect the problem where a hacked client was sending back "untrue" results. Anything that comes back with "different" results gets sent to a third "validated" respondent and the differing on of the initial pair gets demoted from validated. This also solves the workload bottleneck for the time being.
(there are only 4 "letters" in DNA)
<Counts./>
<Counts again./>
Wait a second!
<Counts one more time./>it sure seems to me that there are only three letters in DNA!
</JOKE>
Okay, I don't want to hear any more about SETI@Home not releasing the source code. Here is the real deal on all of this. Do any of you wonder why SGI is blowing the doors off of everybody else? Do you wonder why they crank blocks in under 3 hours? It's not because they have superior hardware, it's because they have the all-powerful SETI@Home client source code. They've gone in and tweaked it so that the FFT (fast fourier transforms) routines use the optimized math libraries (they didn't previously). Bam, instant boost for SGI. Sun has done the same thing, you just can't notice since their average block rate hasn't dropped down yet (they've only been using the optimized client for about 2 weeks now). Both the SGI and the Sun clients had to pass certain SETI@Home tests to be allowed by SETI@Home (they had to crank a block and return results within a certain amount of precision and accuracy).
Both Sun and SGI have offered the optimized clients to SETI@Home to distribute on their home page, but SETI@Home doesn't want them (they don't want forks in the code base or something like that).
That is the link to the alpha binary. Most people probably want the x86 binary which is h ttp://setiathome.ssl.berkeley.edu/software/setiath ome-1.3.i386-winnt-cmdline.exe.
It's just like Linux vs. BSD.. Each side has something they excel at, and something that they lag behind at. Just use whichever one makes you happy.
Phew, I'm not used to waddling around in asbestos underwear anymore .. anyway, I made an update to the disputed editorial at the former location that replies to some of the comments made here and in other forums, check it out if you really care.
Armin Lenz
The Free Film Project can be found off the GNU website. The Internet Movie Project is over here.
It's a small world and it smells funny; I'd buy another if it wasn't for the money; Take back what I paid (SoM)
If you want, you can join in with one or both of the groups I know that are trying to do just this sort of thing. I posted the URLs in the reply just prior to this, and don't want to start getting redundant, here.
It's a small world and it smells funny; I'd buy another if it wasn't for the money; Take back what I paid (SoM)
This is a situation where a hierarchical workload distribution would probably work. Unlike the SETI project with its huge, monolithic data chunks, a spidering project would be dealing with small (comparatively speaking) chunks. There could be several levels of capability depending on host speed, storage, bandwidth, etc. A company with Suns, a few Gig to spare, and a T3 could handle more volume and complexity--maybe spending their cycles figuring out relevance by context and links, etc., rather than spidering. A lot of the grunt work could be done before the cataloged data are returned to the destination hosts. This would also be a little nicer to bandwidth, I guess.
slashdot broke my sig
dang..that is a problem i have not thought of. but i think it is not that much of a problem:
;-)
once someone claims a price for finding RC5 key or next prime or so. one would look in the logfiles and see who has processed that part of the keyspace..and that person would be in trouble then.. especially when releated someohow to the person claiming the prize..
but of course in the seti case that guy could already be hitchhiking through the galaxy and we would not know about it
I guess I don't completely understand this situation, but let me try to pick my words correctly: the Fullon3d web site is labeling Seti@Home an enemy, they want optimized software, and they keep saying Seti@Home is trying to stall. Ok...I still don't get it.
1) I guess we could start by saying the Seti@Home Project is not a video game; it is a distributed computing project connecting hundreds of thousands of computers actively working on the same task. All of this optimization stuff doesn't mean a thing for this project and it's goals. Fullon3d is also reporting that the Seti@Home Project has a lot of competition, which is entirely true. And, as a way to curb this growing *obsession* with work units, they propose the following:
-----
"Limit the submission of units to one machine per email address.
Disallow the transfer of units between accounts.
Retire the Top 20 teams and accounts of all categories every 2 months, that will make it less appealing to push so hard to the top."
-----
Of course, they want to stigmatize competition, but yet, they want to optimize the client to make processing units faster!! Seems a bit contradicting doesn't it? Also, I'm fine running the client software on my P166 comp with 88mb of memory. IMHO, the project is running fine. What need will making the client run *a little* faster fill?
*
2) Everyone has already seen the negative aspects of opening the source, so I don't have to refresh your minds. But, let's think about this situation with some common sense; there is no need to label Seti the enemy.
*
3) Oh yeah one more thing: fullon3d was comparing the Seti@Home project to this:
-----
"How would you feel building a road to the hungry people of the world with your bare hands while the initiator of the project doesn't want you to have shovels because they didn't think to buy trucks and groceries yet?"
-----
What? They are two unrelated situations that deal with different varibles and have no relevance on either of them! The fact is, if you don't want to help Seti, don't download the program.
Rajiv Varma
I didn't think of that possibility. However, if you use sufficient redundancy, then not all of the people processing that block will be running a hacked client. As long as there is one legitimate client which records a positive result back at d.net, then you can demonstrate that anybody else who said that block was a negative is running a broken (hacked) client. It would be necessary to send the redundant blocks to different clients simultaneously in order to make sure that any hacked clients are discovered before the user of the hacked client has an opportunity to cash in on their ill-gotten gains.
You could also seed the work units with positives and check to see which clients record those as negative results, and then ban those clients before they cause you to miss a real positive result. Seeding the work units like this means you have to be able to generate more than one positive result, which would be doable in SETI because a number of different results could be the pattern we're looking for. In other words, there are many possible positive results. Seeded positives might be impossible to use in key cracking because there should only be one key to the puzzle and if we already knew the key, we wouldn't need a contest to find it. The only solution that I can see for d.net would be for the client to not return a positive/negative result but just return the processed block for final interpretation as positive or negative back at the d.net server. Unfortunately this moves back into the "security through obscurity" model.
Your right to not believe: Americans United for Separation of Church and
Well, that's a good point and it would carry more weight (for me) if you ran out of valid work units and your machine powered down. But it looks like seti may be just sending old wu's again to keep people running it, which I agree is a waste.
So, I don't disagree that efficiency is a good thing and that un-optimized clients is a bad thing. I didn't mean to (and I don't think I did) imply otherwise.
I guess I'm just thick because I still don't see how that article hopes to accomplish that. If a 3dnow optimized client appears tomorrow, everyone would still leave their machines on the same amount of time, no? If seti ran out of work units they would likely send out January again. What did I miss?
Ok... I *JUST* got out my ammeter, and my computer takes in .5A. At least in America, we have 120 Volts, so thats 60 watts (if i am thinking correctly here, P=VI, right?) Now, in most parts of the country, energy is about 8-10 cents a kilowatt hour. According to my calculuations (correct me if im wrong here), but thats like 6 cents a night? I'd also like to point out that if a computer puts out any NOTICABLE heat of noise, you probably need a new computer, perferably not an intel *hint*
So gcc, the gimp, gnome, and emacs are all useless for real work? Aside from the fact that there's gnu.misc.discuss for this, what is your definition of real work?
They laughed at Einstein. They laughed at the Wright Brothers. But they also laughed at Bozo the Clown. -- C. Sagan
So there wouldn't quite be an infinite number of computers involved... but computers are faster than monkeys anyway.
--
Win dain a lotica, en vai tu ri silota