Slashdot Mirror
SETI@home: Research on the Research
Officer Larry writes "A professor and a graduate student at the University of South Carolina just posted a study into SETI@home work unit completion times. They came up with some pretty interesting results. It stems off Team Lamb Chop's work, but these guys are interested in how much variation there is in completion times if the same work unit is analyzed over and over again on the same system. It looks like they have other studies in the hopper. Warning: there looks to be quite a bit of statistics in this study..."
Since SETI is most likely purely CPU intensive, there is very little the OS can do to make it faster or slower (apart from deliberately getting in the way, of course). On the other hand, the compiler makes a tremendous difference, and so do the compilation options. Since we do not know which compiler was involved, and which options it was given, the benchmark is fairly useless.
Actually, the OS's can make a difference in performance, even with the same calculations.
Memory management is important for speed, because of alignment issues. OS overhead, for task switching and other stuff matters. Disk access, caching and so on are important. How the OS's handle virtual memory can be important.
But the trials did not have enough samples to fully explore the situation anyway. rbb
...how long till we have a SETI@home@home client to do this research on a distributed level...?
I would have said "it is interesting that Windows NT cannot run a WU at full speed with less than 128M of RAM. Other operating systems make effective use of 128M of memory, but NT needs 256M."
This wang suggests testing linux by running the winnt commandline code via wine! Sure adding a layer of windows APIs on top of linux, that will really improve performance.
Debian 2.2 supports udma100, he should have been runing stable, but in anycase he should state exactly what he installed, and should describe what he removed from cron. A fat desktop system will have tons of programs installed. What he is testing is the performance of setiathome, exim, locate, logrotate, suidmanager, syslogd and half a dozen other programs/scripts all at the same time.
The current Slashdot moderation system is made by gay communists!
Today's vices may be tomorrow's virtues.
Also one thing that seemed to be missing from this article was reference to Roelof Englebrecht's data, collected over some period of time and posted on his SetiSpy web site here:
http://pages.tca.net/roelof/setispy/
Roelof and TLC's Max, are both the keepers of the TLC benchmark database and Roelof checks each and every submission to ensure that the data isn't corrupted somehow due to an unstable overclock.
In addition, I currently run the text client in WINE on my Red Hat 6.2 as standard practice and have submitted data to the TLC page for it. It runs just fine and in fact runs the WU FASTER (whether using the -win95 switch or the -nt40 switch) than either native winblows version when run on the same system via dual boot.
-- Win2k: "It's not so much that it's only 65,000 bugs, it's just that they stopped at 65,535 to prevent an overflow."
Yeah you are right. If linux had won they would have posted it on slashdot or something. Instead we are learning about the study from our psychic friends.
Any alien smarter than us is going to find us before we find them
Isn't that more reason to start searching now? You're right -- aliens may have "found" us long ago. So now it's our turn to find them. Later is better than never, no?
any alien dumber than us (i suppose thats what we are looking for) may not even respond
Respond to what?! SETI is a search. It's not a response. Search first. If you find even a "dumb" alien, wasn't that worth the search? As an analogy, do we search for new species on this planet so they can respond to us? Of course not. They're dumber than us, but still worth knowing about.
As for the fact that it beat Linux, I would guess that the code is optimized for Windows, then ported to Linux.
Also, in my experience setiathome on Linux has blown Windows out of the water...when running without the GUI.
An interesting study, though I'm surprised by their lack of understanding of the memory issue.
Also, they didn't seem to consider possibilities like the fact that a default install of RH Linux may run updatedb daily, which if using a slow drive with a lot of files could easily describe the variation. Instead, their first guess is 'unstable packages.' Wazzup? Academics...they love bizarre conclusions in favor of putting in that extra effort to find the truth. They spent enough time setting up the survey...why didn't they finish the job? (Answer: they were working on a deadline...and didn't allocate time to research the variation)
-zAmboni
-zAmboni
Team Ars Technica Lamb Chop
While we haven't done anything nearly as scientifically viable as this study, our conclusion has been that the difference in processing speed between a 1Ghz and 866Mhz Intel processors (each with 128M RAM) is very slight - almost imperceptible in some cases.
Systems with larger L2 caches, however, seemed to have distinct performance advantages. How (or if) you can apply that information to non-SETI applications I will leave to you.
-Coach-
Perhaps the world's greatest tragedy is that ignorance is not impotence.
I can't believe they didn't test any Mac platforms. Mac users are some of the most enthusiastic fans of SETI! Team MacAddict is #3 in the Club Competition , only behind Ars Technica and Team Art Bell. If fact they're almost tied with Team Art Bell, with only 5,000 users to Art Bell's 13,000 users !!
It's good to see such a strong response from the community and we thank everyone that has given us constructive feedback. However, we feel that many of you have completely missed the purpose of the article.
The purpose of the article is NOT to compare OSs, get fast SETI@home times, or even to optimize the systems at all. We thought this was clearly stated in the abstract and methodology sections. Either we screwed up and did not make this apparent or people did not read the article. The study of optimizing systems and making SETI@home faster is already handled very well by Team Lamb Chop (if the site ever decides to come up...).
What we DID want to accomplish is to study work unit completion time VARIATION . This question was prompted by our reading of the SETI literature which had not dealt with this matter. Our research question was: on a "typical" system, how much does the completion time vary from instance to instance? If I run the same work unit over and over, how much will my completion time vary? I don't know about the rest of you, but when I run SETI@home on my systems, I do not disable all other processes (including cron or anything else). If we did disable anything out of the ordinary, then we did not accomplish our goals. As was clearly stated in the methodology section, we used the default settings for each OS.
A separate question was the method used for us to arrive at this conclusion: testing several OSs in several configurations on the same platform. At this time, this method seems a useful one for comparing configurations.
Now that we have a baseline, we are going to continue and try to figure out the questions posed in the article. Such as: why does Linux have such a high variance in comparision to the other OSs? We have collected many great suggestions and I will test all of them and post the results.
In any case, this article cannot be used to compare OS to OS performance. Not only is that not the purpose of the article, but there are too many confounding variables--many of which have been raised here.