Slashdot Mirror
SETI@Home Says Client 'Upgrades' Are a Bad Idea
bgp4 writes "New Scientist has an article on how 'upgrades' to SETI@Home clients are causing some trouble. Even though the upgrades speed the client up, SETI reps don't want people using them because they may induce bad data. If SETI@home just open-sourced [the SETI@home client], they'd have better PR and a better client." Amen! SETI@home, are you listening?
This is a scientific experiment. It's not a race, it's not a demonstration of the power of computers or computer communities.
Comon people, think about this before you flame. Someone submitting invalid data can ruin this entire project for everyone. Open source software doesn't get tested until it's too late, and by time a calculation bug is found in the code, thousands and thousands of entries could be invalid. At least think about the project some before you randomly spew out "Free The Source!!"
Yeah, I run seti@home. Yeah, I'd like it to be open source. But it's not.
If you agree to help out, you do so on seti@home's terms. They say they want you to use this software, so you use this software.
This is not about having the most units completed, or about being the one to find the signal, or about improving the software. It's about helping the project, and contributing to the body of scientific knowledge. If you want to help, use the sanctioned software. If you use anything else, then you're hindering. Go crack cyphers or calculate weather patterns or something.
But give me a break. Stop antagonising Seti@Home because they're doing it in a traditional way. Who cares? They've come up with a great concept to advance their research, and a lot of people bought into it. It's not like they're making money out of it. It's not like the publicity they get fills the bank account of a rich CEO.
So stop picking at Seti@Home all the time until they become Open Source. Change is gradual, and we have to count or blessings and support Open Source initiatives when we see them, not declare a Jihad against everything non-OSS. Yes, most people including me would like to take a peek at the code. But let it be, for just a moment... Seti@Home remains a great project, whether it's Open Source or not.
You know, there *are* great software that don't need to be Open Source... And I think they're not hurt that much because of it. I don't think Seti@Home would get a whole lotta good from Open Sourcing their code, because everyone could figure out how to send false data claiming they not only got an E.T. signal, but it was singing Singing in the Rain in slow-motion.
"The wages of sin is death but so is the salary of virtue, and at least the evil get to go home early on Fridays."
SETI is listening, and your arguments are rejected.
This may sound odd to people participating in distributed.net, but SETI is not about processing data as quickly as possible. It's science. In science, you want to hold as many of the variables as similar as possible, so that you can be sure they didn't create a false result (false positive OR false negative). All else being equal, speed is nice, but it is not the goal.
Open source is not the answer for everything. Sure, if it was open source, some good patches might come out, but how many people would download the code, apply the patches that speed it up, and never have a clue that they just fatally broke the FFT result testing algorithm? Or for that matter, if they broke the FFT algorithm? Or would simply use it to easily learn how to send result blocks without processing them?
The fact of the matter is, even if you can improve the code, you cannot improve the code. (That's not a typo.) If you can improve the code, instead of helping SETI by processing keys faster, you bring yourself out of alignment with everybody else, create potential bugs in the experiment, and render all of your results suspect. SETI is science... distributed.net is engineering. There is a big difference, and science does things the way it does it for a reason. SETI needs the results to be as solid as possible. (If one of the hacked clients detects a signal, rest assured that even if SETI doesn't subject it to extra scrutiny as a result, some other scientist will.)
SETI can't stop people from modifying the executable on their own systems, but I think the people calling for SETI to make it even easier for people to modify the system (not just your code, SETI is part of a system and subject to the interactions thereof) have a fundamental misunderstanding of what SETI is about.
But, it *is* about finding the signal. It may not be about *you* finding the signal, but the *whole point* is to track down a signal.
To track down the signal, units must be processed. If we process signals faster it'll either let them eventually process more signals (larger spectrum, etc) or use less computers. Either is a worthy goal. One helps science, the other simply saves power and lets people run other distributed projects.
SETI@Home is hindering their own project by insisting on slow (They refused AMD's offer of help to speed the client up, doing more work, as carefully as before, in less time.) clients just to keep a large number of users helping is a bad PR move. You could help a project that wants to process data, or you could help SETI who wants a large number of users, who incidentally process some data.
The insistence with which some people clamor for open sourcing everything really annoys me (and a lot of other people). There are very good reasons not everything is open sourced, and sometimes they're not even due to stupid licensing restrictions imposed by third-party code.
For something like SETI@home (or distributed.net or whatever else you like), there's a very good reason to keep the clients binary-only. Namely, there is no oracle for verifying that a block of search space was actually searched by the client that claims to have searched it. Abuse of this was seen by the DES challenge and distributed.net before; open-sourcing SETI@home would lead to even worse abuses. Unethical people would modify the code to claim they had searched oodles of key blocks, ruining the results of the search -- and only so they could show off how "studly" their computer system is.
Of course, maybe this concept is too hard for bgp4 to grasp. But for goodness's sake, it's in the SETI@home FAQ. Whining about their policies on Slashdot isn't likely to change their minds.
(Beyond the malicious introduction of false reports, it's very easy to "optimize" something like this and introduce numerical or algorithmic errors. Unless you are familiar with advanced theories of signal processing -- the sort of thing you'd find in graduate classes at a good university -- you would be well over your head in looking at how the algorithms work. And there are enough bright grad students working on the average project to know how to optimize for all sorts of cases without the help of a bunch of open source zealots who think that the GPL is some magic potion that can be applied to anything to make it better.)
No CVS was ever set up (though it was repeatedly promised, those times any response could be elicted at all!), and the frustration of those volunteer developers taking part was very evident on the mailing lists.
There were a grand total of SIX snapshots ever made. SIX! At the very least they should have been daily!
I'm not the least bit surprised that the frustration with a slothful and (frankly) incompetent development team has burst out with "home improvements". That was only a matter of time.
As for SETI@Home's "reason" for not releasing the source - the need for a secure, validated connection - well, I think that's been blown out the water. If their connection was so secure and validated, rigged clients wouldn't be possible. The fact that SETI@Home are complaining at all shows that they're no better equipt to secure a connection than the movie industry with it's DVD encryption.
Let's all cut the pretense, and stop this nonsense. I want the very best SETI@Home clients that modern computer skills can create, looking for real data, using real analysis, not cheap junk that goes faster because half the tests are skipped. (Notice the gaussian stats vanished with the first official "faster client"?)
Now, SETI@Home obviously also wants genuine security. So put your source where your mouth is, SETI folks, and RELEASE THE CODE! Your attempts have failed, miserably, so let those who know what they're doing take over!
It's a small world and it smells funny; I'd buy another if it wasn't for the money; Take back what I paid (SoM)
If people would read the FAQ on SETI@HOME it tells why they are NOT open sourcing it. If the client is open source then the data that is sent back to the servers can be faked, and even if it can be faked that calls the whole study into question since anyone could send back a bad packet to get points (and it would probably happen) and they would have no way of knowing.
You're missing the point. People are faking data NOW. It's done already. The client has been hacked.
Open source is not a lack of security, it's a big plus. If they want a way to make it much more difficult to fake packets back, why don't they talk to d.net? I know d.net had to deal with that problem in the past. So come on already.
I agree that you mainly want to be sure that the data is valid, but that does not preclude optimizing the client for speedups. They gave the source to Intel to optimize for P2 and P3's already. They won't give it to AMD (or haven't yet). They're deliberately withholding it from people who can make the improvements, and do it securely.
All it's really doing is FFT's anyway, which really lends itself well to optimization on various CPUs. Just a load of floating point, although 3dNow! instructions probably won't help much, since those are single precision only, I think. You'd want double for something where you have that many significant bits.
---
- Give a man a fire and he's warm for a day, but set him on fire and he's warm for the rest of his life.
What we see here is just one more guy who simply cannot believe that open-sourced software is more secure then proprietary one (Just like the old argument of some people - mainly the less mathematically knowledgeable suits - who can't believe in the security of OS encryption software, after all the algorithm is readable by everyone and therefore unsecure).
In reality, the original patch from Olli was a good chance for moving in the bazaar scheme not only to processing power but also to client development, since it was a simple wrapper which didn't do anything special itself, but enabled other programmers to write their own Fat Fourier Transformation algorithms for special chips.
If S@h would have allowed patching the software (or had even made the client OS) we could now have dozens of different clients tuned for every imaginable Streaming Extension, DSP or multiprocessor enviroment. And the SETI@home project could have also been a starter for future similar projects which could have used all the power of the specialized out there.
Now for the defense of the project heads: There have been multiple malicious tries by individuals in the past who tried to get their rating up by cracking the protocol and sending bogus data to the server. This perhaps explains why they're this paranoid.
I wanted to take a look to see exactly what the heck these patches do, but I'll be damned if I can find any. The best of search engines (several), and I can't find them anywhere. First time I've found myself unable to find something like this.
Any tips, anyone?
---
- Give a man a fire and he's warm for a day, but set him on fire and he's warm for the rest of his life.
I'm seeing comments saying that SETI@home wants accurate results, not speed. Someone actually posted that "improving the code won't improve the code." That if you try to make it faster, you are out of line with the SETI@home's goals.
How can I say this? Well let's try this: what the HELL are you talking about?
Please, let's *think* here. Why are the goals of accuracy and speed mutual exclusive? Perhaps there are some methods that would reduce accuracy for speed. Fine. Let SETI@Home set there tolerance levels (i.e. how much significance to use on calculations, precicion, etc.). Then let people who are very, very good at making C programs run really fast and making programs run reliably and accurately take a crack at it.
Come on people. There is no reason that SETI@Home could not be improved by an open source approach. I agree that accuracy should not be sacrificed. But pardon me if I think the resources of the entire program world might not be able to produce a faster and more reliable/accurate client. It's just a matter of raw resources. More programmers, more peer review==better, more reliable code. And it just might happen that it comes out faster in the process
ranting ends ...
"Doubt your doubts and believe your beliefs." -- Switchfoot, Ode to Chin
The reason is, as someone has noted, that I can FORGE the packets so that they will look like normaly processed data, whle they will be bogus. And I know already some people (by their nicknames) that have tried to do this, and get a high rank. I have seen people processing thousands of packets in an hour, while, with the official client, you can process a packet in 9 - 10 hours on a P III 400 MHz (running NT or Linux, no big difference).
Sigged!
SETI is science... distributed.net is engineering. There is a big difference, and science does things the way it does it for a reason. SETI needs the results to be as solid as possible.
The main thing with the SETI client _should_ be not to make sure it finds a signal, but to make sure it doesn't miss one. I agree with this part of it.
However, this is data analysis. Pure and simple. Run an algorithm on a lot of signals. Easy. No one questions the algorithm. What is questioned is the implementation. If I can take an algorithm and optimize it to run faster on a particular processor, then I must still get the same results. Otherwise, the algorithm is no longer the same.
And it's not a question of some hacked program finding a signal. ANY signal found will be subjected to the most intense scrutiny even before it's announced. Then it'll be scrutinized again afterward. And other signals from that sky area will be looked at, at various times going back years. No, a false signal will be eliminated pretty quickly. The thing to watch for is a false negative.
I don't think open source is the cure-all answer here, but I think the people running seti@home have not given thought to the fact that people running the client are the kind of people who really, really know computers. They know algorithms. They know the internet. They are smarter than the average bear. And they don't like people telling them you cannot know this, or you cannot know that. The SETI people need to explain their position better, or count on a lot of people leaving the project.
Just my $0.02
---
- Give a man a fire and he's warm for a day, but set him on fire and he's warm for the rest of his life.
but enabled other programmers to write their own Fat
:o)
Fourier Transformation algorithms for special chips.
I believe you meant Fast, of course, but this little typo is really cute
And who knows, maybe the good 'ol frenchman was actually fat and didn't mind it a bit!
To stay on topic, here is a link to clues about Fourier Transforms.
Sigged!
In the case of the SETI@Home project, the security is needed by the project, who don't get to see the modified versions. So it doesn't apply.
Of course, if people distribute their modified versions, other people will find the bugs. But how will the already-submitted results be retracted? And what about modified versions that people keep to themselves?
I suspect that future projects of this kind are going to have to include more dummy data to verify that incorrect algorithms are not being used.
Seems to me that the Seti@Home people are caught between a rock and a hard place. In order to attract lots of cycles, they've made a contest out of it. That's what posting team rankings on number of results returned amounts to, after all. And that inflames the folks who know they could rise in the rankings if only they could optimize their clients.
On the other hand, they prize uniformity of results over speed. Hmmmm.
Well, if they already have too many volunteers, the answer is simple: dump the contest and make it a lottery with one winner. No more rankings, no more teams. Just tap the guy who finds the aliens.
All the contesters will drop out and go back to cracking RC5. The hard core will remain, consisting of the ones who agree that uniformity is more important than speed, together with the ones who seriously want to mess with the project. There'll still be the need for security but at least most of the results will be consistent.
First of all, where exactly do you find this "official" slashdot sentiment. I looked everywhere and I couldn't find it. I'm sorry, but the very fact that you posted this article shows that there is no 'official' slashdot sentiment since you are a slashdot reader and you have a different opinion.
You mention that people have already reverse engineered the code. Well then, this already shows that even if they don't release the code, the security is bad. Open source could actualy strengthen the security as I mention below. You conclude that releasing the code would only decrease the security, but how do you know that? What if for every one hacker that is out to screw up the code, there are ten programmers that can improve the code. I don't see an obvious outcome of this being that the security is lessened.
I suggest that some method of redundancy be introduced so that more than one computer be used to analyze the same packet of data. This way, even if one guy does fsck the data up, there are a number of backup computers to double check it. This is one of the fundamental principles of science, redundancy and reproducibility.
They are only making about 1/3 as many as are sent out. In addition, they only have a limited time (2 years?) of data gathering time allocated to them. If we go any faster, we will just get more repetition in the packets sent. We can't find ET any faster by speeding up the clients. The only way to go faster is to collect more data from the telescope. Unfortunately, the S@H site tells us why this is not possible. Maybe we should all read the info available from S@H before going on and on about the "inefficiency" of the clients.
Creepers, they are asking for our help with their project and we make it sound like they should be flogged for not being "perfect." If you don't like it, don't participate in it.
--
--
dman123 forever!
Filtering out the -1s and 0s since 1999.
While I agree with you, I think you, and a lot of the other "Open Source SETI now" zealots have missed the point. This has more to do with scientific method and the ability to convince other scientists and observers than it does technology, algorithms, open-scource and computers. Seti@home is an experiment, not a software package...it just happens to use software to do the experiment.
If a signal is found, your right, it will be analyzed to death before it is announced. False ones will be rejected. But with any experiment, you must be sure that ALL conditions are the same everytime it is repeated, to be sure the results are as accurate as possible. If the source is opened, they have just lost that control...they cannot be sure (100%) that the analysis is repeated under the same conditions. As we have already seen, some people have tried to boost their scores by sending the same packets over and over or by sending false packets. While these will get filtered out, the project can still say their results are trustworthy because all analysis is the same - same implementation, same algorithm. Imagine if it is opened up and a signal is found. It will be processed, double checked and checked again. But when they announce it, there will be a large number of people (and scientists) who, for whatever reason, will refuse to believe it. They will then point to the lack of control over the source as a place where the data was either " messed up to give a false positive" or deliberately faked. As they say in court, they will have a "reasonable doubt". This is too important to have that kind of possible outcome.
Put it in another context - a group is experimenting with an AIDS (or ebola or something like that)vaccine and testing it in petrie dishes (this is an example, so excuse me if this doesn't really happen). They set up a bunch of dishes they had control over (washed and prepared themselves). They get some help from people outside the experiment who can make the dishes clearer or easier to handle or what ever. They check and double check and find that in a lot of the dishes the virus is killed quickly and in some it is not. How do you interpret these results? You don't - the experiment is invalid because you cannot safely say which pertrie dishes are which and don't know wether the results are due to your drug or due to some bleach left in the dish by those who prepared it (by accident or on purpose). You can only be sure about your dishes but you can't tell which ones are yours. The solution is to only use your own dishes.
I'm sorry for being long winded but I'm trying to stress a point. So open source can make the client more efficient and faster, who cares? Being sure of the results is what matters.
The sad part is these patches may have already made the experiment invalid for thse same reasons...considering the size and reach of the net, who knows how many "enhanced" clients are out there. It is possible that, if a signal is found (or missed, in which case we may not realize it) that all the redundancy checks were done on clients with the same or similar patches with the same calculation flaw and all becasue the project doesn't have complete control...and because some 19 year old wanted to be #1 on the SETI@home list.
Hey, if you don't like it, don't use it. It's their experiment we must participate on thier terms.
Never by hatred has hatred been appeased, only by kindness - the Buddha
Why are the goals of accuracy and speed mutual exclusive?
Well, here's a demonstration you can try at home. Find a recipe for, say, chocolate chip cookies. You want more speed? Double the heat setting on your oven, and cut the baking time by half. Watch what happens. Your output is no longer accurate, even though the input (yer ingredients, order in which you combined them, etc.) is the same as what was called for in the original "source".
Now open-sourcing recipes is a fine idea. Go ahead and experiment in the kitchen, and if you can come up with a faster way to make cookies that taste as good as the slow-cooking ones, more power to ya. But don't expect Betty Crocker to print your recipe in her next cookbook until she gets to test it out herself.
The folks at Seti@home might be better served if they open-sourced their code. It seems like a good way to improve it. But one programmer's improvement is another programmer's bug. And if someone's "improved" Seti@home code is fast but sloppy, and gives unacceptable results, the folks at Seti (and all of us who care about the project) lose out big-time.
It would be nice if the code were available to be tampered with, fine-tuned, and "improved". It would also be nice if only "real" improvements - not quick'n'dirty shortcuts - were used in crunching the data. But how to tell? We don't live in a perfect world. Open the source and big improvements - as well as tiny-but-devastating bugs - may follow.
There is supposed to be one accepted program for crunching Seti's data. Arrange it so several versions are running, and you introduce more variables into the experiment. Not good.
>SETI can't stop people from modifying the executable on their own systems,
:)
>but I think the people calling for SETI to make it even easier for people to modify the system
How can closing the source prevent me from spoofing a valid signal with a client I write and distribute? Nothing! In short you can't ensure that you are getting valid results back simply by saying no one can know HOW your client works because you can't control the source of the packets. I can write a bogus client that mimics your interface.
There is no inconsistency to having the client open, and then still ensuring that only valid clients be used-these are separate issues entirely.
According to your statement I quoted above, SETI's experiment is already blown if they can't stop people from modifying your executables, so why do they continue? Because they (I hope) have checks to see that the data is valid.
Point is, no one has to hack the closed client they can write their own. The only way to secure the client is to find other ways to validate the data. Thus validity has nothing to do with the openess, so open it and you gain the insight of the best programmers in the world both on performance, and probably security too.
This isn't about speed, or accuracy in the Scientifc processes that SETI is using, if it were, then no Scientific endevor ANYWHERE could use open source software. Guess all those universities will have to get rid of Linux
I don't think anyone is advocating that SETI release control of the project, just suggesting that we leave the astronomy to the astronomers, and the programming to the programmers.
>If you can improve the code, instead of helping SETI by processing keys faster,
>you bring yourself out of alignment with everybody else, create potential bugs in >the experiment,
No one is saying that giving any goofball the ability to screw with your algorithims will help you, but they ARE suggesting that you could get a lot of help with the valid client that YOU distribute/endorse. If you choose to abdicate that responsibility, well than you are no better than the suits who think that Open Source is a panacea that means they don't have to work anymore either.
Security through obscurity, you'd think the scientific community would know better.
We are agents of the free
RC5 has a known number of keys to check, and that number is truely enormous when compared to the number of keys per day that can be done. Thus, there is every incentive to optimize the hell out of the client.
By comparison Seti@Home is already processing data faster than it can be created. Optimizing the client won't make the analysis go any faster - maybe they would be able to check each work unit three times instead of two. But, where's the incentive to optimize?
S@H is a victum of its own success. What will probably happen is the number of users will start to drop off as people become bored. This isn't necessarily bad.
...phil
...phil
"For a list of the ways which technology has failed to improve our quality of life, press 3."
- People are conditioned to want their computers to run faster. The amount of time and effort some people spend to overclock and benchmark their computers is often far out of proportion to the actual benefit they get from their computer's speed. It's not surprising that people treat their SETI@home processing speed as a benchmark.
- The fact that SETI@home puts up statistics that have turned this experiment into a competition to complete the most work units reinforces that behavior.
- At least one company (perhaps SGI, but I can't remember for sure) has mentioned their SETI@home crunching speed in some marketing literature, again emphasizing speed over quality of results.
- As several in this discussion have pointed out, making the clients faster won't help the project because the bottleneck is that SETI@home can't prepare the units fast enough. However...
- If the client software were improved, clients could potentially do more sophisticated processing in roughly the same time, improving the science. However...
- This could make the clients seem even slower than they already are, which wouldn't sit well with the kiddies who are more interested in their rank or how fast it makes their box seem than the science involved.
So what lessons could be learned if this or a similar experiment were to be done again?- Deemphasize the ranking of work units completed. Perhaps if the concept of a fixed work unit could be dropped altogether (i.e. make the "size" of a work unit something arbitrary so that they couldn't be compared). This would possibly prevent the client from being used more as a benchmark than for its true purpose.
- Plan for hacked clients and spoofed results by sending out enough test work units and by cross-checking results with multiple clients enough to have confidence in the results backed up by statistics.
- With enough cross-checking, you might as well Open Source the client.
I would be interested to hear if there is a (theoretically) foolproof way to use distributed clients to produce results with confidence if you accept that some clients will be spoofed.Point well taken.
Of course, what does this have to do with Open Source? If you don't like they way they do it, create your own version. I may not like Word, but I'm not demanding MS open source it, I'll use KOffice, StarOffice et al or create my own. Let them identify the slice their way first then check it out...don't mess with it midway through the experiment. Your proposal is to do the experiment a different way. Do it, but changing the experiment half-way through makes no sense.
And I say once again:
"Hey, if you don't like it, don't use it. It's their experiment we must participate on thier terms."
Never by hatred has hatred been appeased, only by kindness - the Buddha
Uhm...then don't run it. Your choice. Me? My machine behaves just fine and I don't care how fast my work units get processes. I'm doing this to help out a program which lost all its funding in 1993 because some senator couldn't get a Radio Telescope in his constituancy (or some other equally stupid reason). This is for the science. If your moral code doesn't allow you to run it, don't.
Never by hatred has hatred been appeased, only by kindness - the Buddha
No they don't. I'm still using the first client on my Win 98 machine and have 29 units complete and counting...they're being proicessed this very minute.
Never by hatred has hatred been appeased, only by kindness - the Buddha
Open-sourcing has the further advantage of becoming very, very portable, almost for free. I'm sure there are a few bored mainframers out there with a few underutilized MVS boxen that could contribute to the Cause.
<troll>But all of us know the real reason they won't publish the source--SETI@Home isn't sending us space noise at all; they're sending us heavily-encrypted high-level diplomatic radio transmissions from all over the world, letting us do the CIA / NSA / NRO's dirty work for them. Until we see the source, they can't prove it's false; therefore we must assume it's true until we see the source.</troll>.
Has anybody seen the latest DNRC office prank? Someone's co-worker was running SETI@Home on their office PC, so the DNRCie hacked him a screen saver that beeped and kept flashing the words SETI@Home: Possible extraterrestrial transmission found. Confidence 99.9893% over and over again. I can think of a half dozen people I'd like to do this to, only I'd make damned sure that on the next user input event, they'd get a dialogue saying "Please stand by: writing results to disk. DO NOT INTERRUPT THIS PROCESS" followed two seconds later by a dialogue box saying "Fatal exception error", because there's no point in being just a little bit cruel.
--
This is not my sandwich.
BTW, for all the "why not Open Source" whiners out there, 18 months ago, this project WAS open sourced. You could volunteer, talk directly to the project leaders and hack the code all you wanted. This was during the DEVELOPMENT phase. When the development was finished, the client was closed. Where were the whiners when you could get the source? I got 2 different versions of it to play with (though I don't have them now, since I've nuked my hd a few times since then). The client is NOT a commercial product. Open source is great for commercial products because it continuously improves the quality of the code and design. A commercial product needs this improvement to stay competitive. The SETI@HOME client does not need this. It is doing what it was designed to do for this experiment just fine. There is no "competing product" to stay caught up to or surpass. In 2 years, when the experiment is over, so will the SETI@HOME client. Therefore, any of the benefits of OSS development have already been utilised by by SETI@HOME. It doesn't need to be tweaked or spead up or become more efficient. It simply needs to be used by 'volunteers' the way the project leaders have designed it. If you can't abide by their request, don't volunteer. If you don't like the way it works, don't use it.
If SETI@HOME didn't have their ranking scheme, would we be even having this discussion? I think that was their only mistake....
They already used the Open Source model...
Never by hatred has hatred been appeased, only by kindness - the Buddha
In the case of SETI, there is a mismatch between the potential developers and users that is unique to a distributed system. The client user/developer wants a client that is faster, potentially at the expense of accuracy. The official SETI developers are not users primarily, but instead want to have great confidence in the client, potentially at the expence of speed.
This mismatch turns many of the benefits of open source on its head, from the perspective of the SETI developers, since their goals are different than the potential developer community. Note that the client user/developer would achieve his goals quite well if the client were open sourced, as per dogma.
Perhaps we need to tweak the dogma a bit to account for this bizarre bazaar. Instead of "open source is always better", try this on for size:
It does not necessarily make me happier with what you are running on your computer. If you are running a server I am happy that the server is open source because that means it has open interfaces. I'd be equally happy if you just opened your interfaces. If you are running my code for me in a distributed type of network, I want my goals to be preeminent in the code design. Open source only gaurantees this in the rare case that our goals completely coincide.
For open source to work reliably in a distributed network of users with varying goals, there has to be a central authority with approved clients for whom the cost of approving clients is less than the benefit of the potential innovation. In addition there must be a way to strongly authenticate that those clients have not been modified. Incidentally, an authentication method is what is really needed in a closed source case as well. I can't think of a foolproof method for doing this, the best bet is probably just random result comparison and some speed heuristics.
So the answer is that SETI could open source their client, and the people running it would benefit. But it might not be a benefit to them since the benefit of added speed (the most likely outcome) is not currently needed, and the potential for good faith bogus clients increases greatly. Bad faith bogus clients would likely stay at the same frequency.
--
"L'IT c'est moi!"
The current policies of SETI@Home are clearly wrongheaded; security-through-binary-obscurity is not the right way to protect the integrity of their project, and they're falling into the same close-minded trap the DVD Forum did. Obviously, they should open source the client (and the server!), but maintain an "official" binary distribution with an embedded signing key; their servers would only accept input from blessed binaries. (Yeah, it's possible to extract the key [though this can be made difficult]; you'd want to send a different key with each download, so you could easily revoke one. The point is that this policy would allow well-meaning contributors to play with the client source and submit optimizations for review.)
Furthermore, they'd get people making not only performance improvements, but bug fixes. Do they really think that by working alone they have better QC than if everyone could review the code?
That's not my point, though.
Normally, when the community at large is faced with someone taking a clearly broken licensing approach, we can react by making our own version free of encumbrance. Here, however, we're tied to SETI's data source -- few of us own large radio telescopes.
Is it possible to gain access to SETI's raw data feed? (Of course it is; they send data blocks to everyone for processing!) Is it possible to come up with our own code for analyzing that data and processing it elsewhere -- with or without the cooperation of the S@H administration? I don't think they'd be happy; but could they stop it? (Surely not without causing even more ill-will.)
...
Tangent: What if the ETs use ultra-wideband (UWB) techniques (see http://www.uwb.org/, http://www.time-domain.com/)? Won't FFT-based analysis totally miss such signals?
There are some side effects of open-sourcing the SETI client. As I understand things, the SETI client periodically transmits data to an outside machine. If SETI were open-sourced, it would be easier for rogue programmers to alter the source to send files (or whatever) to a programmer. Sure, we would all have the source. How many of us actually look at the source? Furthermore, a less than scrupulous programmer could publish source that didn't match the compiled binary.
I am somewhat concerned that the SETI project could be a front for the NSA and/or Echelon. How can the NSA keep up with analyzing all those voice lines they're monitoring? Why don't they just right a cute little screensaver that analyzes voice lines, and transmits the results back to home base? Hmmm. If that's the case, they really don't have anything to benefit from open-sourcing the code. It is this that prevents me from running SETI on my home machine...
--Be human.
SETI has always had trouble creating enough work units to keep up with user demand. The project is more popular than anyone had ever dreamed.
Look at the date of the data you're processing. The date of a work unit I just downloaded right now is July 20. Analyzing this data isn't supposed to be fast. It's supposed to be accurate. It's better to do something slowly and do it correctly than to hurry through it and find out you did it wrong. I'm not saying that SETI couldn't make their client faster, but the idea of it going too fast is ludicrous. They're still sending 4-month old work units. IIRC, when I first started SETI@Home, I was also getting 4-month old work units.
SETI just can't generate that much data to munch. If they did speed up all the clients and they ran out of blocks, users would be very upset. Solution... SETI would start sending out the same blocks to be rechecked while waiting for new blocks to be created. And users would be upset because they were doing the same blocks twice.
This is science, not a game. Scientific data needs to be checked and rechecked many times. The problem (from the article) isn't because of SETI. The problem is because of people who believe that distributed computing is a race instead of a way for organizations to effectively speed up processing of data. If results aren't accurate, the entire process is useless.
You want more speed? Double the heat setting on your oven, and cut the baking time by half. Watch what happens.
That's a great example, but it is also a straw man. That example works for cookies, but not necessarily for software. I agree that there might be a way to speed up the code at the cost of accuracy. But that is not what I am advocating. I'm advocating better written software--which will mostly likely be at least of of the following: more stable, more robust, faster. Better written, by definition, has to be at least one of those. And we already have a requirements for the project: no loss of accuracy. Fine. We work towards that goal. That's what software development and design review are all about. Open source just taps the resources of thousands for those tasks.
And just because they open source it doesn't mean they lose control. As these post evidence, many who do SETI are doing it for the right reasons. So if SETI realeses the code FreeBSD style--i.e. where they retain control of the "official" tree--then they gain all the benefits of opensourcing along with maintaining specific control of their research. The authentication problem could easily be solved by some public key system that is controlled by SETI. And before we start arguing about that, let's remember that they have already LOST that battle. So at this point, an open approach to how to improve the client would only help them.
Again, cookies are one thing. But software is another. I am sure you have worked on projects where someone wrote something accurate, but slow. Perhaps their memory management wasn't right, or they were using a slow method to get something done. The code worked. The code was accurate. But it could be improved and remain accurate just by having someone more experienced look at it.
And that is what I am advocating. Remember, that's the whole open source "thing". I think your confusing GPL, forking, and crab grass development techniques with a time honored scientific approach: peer review.
When it gets down to it, that is EXACTLY what open source is all about.
"Doubt your doubts and believe your beliefs." -- Switchfoot, Ode to Chin
I understand the merits of keeping the source to seti@home closed. It would be quite aweful if the project were invalidated by bogus results. However, I have a real problem in that the project maintainers seem to want their clients to run slowly. That's right! they seem to want to run inefficient code on your computer.
My chief example of this comes from some work that compaq has done on the client. They asked for the source to Seti@Home so they could get it to run on Alphas. the sent their version back to the maintainers and that is the alpha client that is being used now.
However, later they went back and tuned the client to run more efficiently. They linked the seti@home code against their Compaq Extended Math Library which contains a highly optimized FFT routine. The results are stunning. If you check out Compaq's stats, they have 21164 Alphas completing data blocks in under 3 hours. On my 21164 machine the best I've ever seen is 6. The speedup on the 21264 processor is even more dramatic. Check out their top users in the compaq team, they're flying through the data in under an hour per block!
Seti@Home has recieved that patch many months ago. (I found out about it from a compaq engineer on the now dwindling AlphaNT mailing list this summer) However the package maintainers have done nothing. This leads me to conclude that they either are technically incompetant and lack the skills to test this enhanced client (which I doubt because they are letting compaq use it) or simply want the clients to run slower.
So while I agree that open sourcing the client would be a very bad idea, I fail to see the logic behind refusing to distribute perfectly valid patches that could INCREASE the amount of science that seti@home does.
Having worked with the gcc compiler for many years I can say that math fluctuations are increadibly easy to introduce through optimization. Floating point operations, though rarely used in CS education, are rampant in the Seti code. The problem isn't in producing bad data but in differences in the output that most floating point optimizations produce. Floating point operations are much more prone to fluctuations due to optimization. The optimized seti clients are guaranteed to produce different results than the original clients.
Since they haven't been able to collect enough data by orders of magnitude to feed their clients, there's little use to optimized Seti clients outside of learning how the Seti client works.
What if Seti used some sort of checking to verify data? Example:
Person A processes the data and checks the processed data into the server.
The data person A had is assigned to a (random) person B. If person B verifies what person A found, the data is accepted to be true.
If the data person B processed is different than the conclusion person A came to, the data is sent to person C and D for processing. C and D should verify what either A OR B got.
Of course, this could cause problems if abused, but it would allow for an open source client in that nobody can REALLY forge data because the data would be processed more than once. Of course, one may argue that this considerably slows down the effort .. but this is the best way to operate in a way that may lead to better efficiency.
But then again .. what do i know, and why are you reading this?
Personally, I don't believe most people are running it that way. This one doesn't work like RC5, or the Mersienne Primes, where it can sit quietly in the background. Since the S@H client takes over so much of the machine, I believe most people only run it when they aren't using their computers. Thus, it's not a question of "doing other things with the computer". When the user is doing the other things, the client is basically shut down.
It's exactly the But from the _user's_ viewpoint, it's taking him hours to do one data block, it should be faster! viewpoint that leads to the race mentality and ultimately cheating to get higher in the rankings. The important thing is the science, not the race. I've got several machines running the S@H client, and I quit checking my rankings a long time ago.
...phil
...phil
"For a list of the ways which technology has failed to improve our quality of life, press 3."
And you're absolutely sure that when you swap in this new FFT subroutine, that you will get the exact same results, down to the least significant bit, every time? Because if you're not sure, then you've effectively changed something in the project, thereby throwing the entire thing into a realm of uncertainty that science does not handle well. It could be enough to invalidate the entire project.
...phil
...phil
"For a list of the ways which technology has failed to improve our quality of life, press 3."
What about the person who hacks the client, not realising he's broken something stopping potentially valid results from being sent back. Do we then check all negatives too? If so, why bother?
Yes, a few people who use S@H are "smarter than usual", the kind of backpatting people occasionally like to give themselves. I have a reasonably high IQ. I *don't* think I'm qualified to fiddle with algorithms people with PhD's in astrophysics have spent *years* on. "Thousands of eyes" don't help here.
Open Source. Closed Minds. We are Slashdot.
Because the problem, as defined by the Seti@Home team, is not infinite in scope. In fact, it's very finite. The are only generating about 30 gig of raw data per day, and the analysis capacity in place is way more than sufficient to analyze all that data.
Now, you might say "expand the project". That's a good idea, but it takes money and people that are not available. The S@H guys are running this thing on a relative shoestring. If you are going to tackle an infinite project, you have to throw infinite resources at it, and that's simply not available. They are doing what they can with what they have.
...phil
...phil
"For a list of the ways which technology has failed to improve our quality of life, press 3."
Te obligatory this isn't news reply:
SETI said the same thing about releasing the source to the 3Dnow team and they certainly aren't going to release it to the public now.
Personaly, I think its a major waste of computing power when other projects like GIMPS have 30,000 members while SETI is pushing half a MILLION.
The Spielbergesque lure of meeting ET keeps morons happy with their number crunching competitions while real science like GIMPS is mostly ignored.
Now the number crunching competition has gotten to the point where people with a lot of time to kill have written patches to have the fastest team on the net.
Its the geek equivilent of drag racing. SETI open source advocates should admit they're in a bloated project, swallow their BS techie pride and crunch slowly away for science or go do something else to show off.
Not true. There have been several groups making requests to the SETI@Home folks to add processor optimizations. The 3DNOW camp of AMD zealots even rewrote the transformation routine used by S@H and submitted it for inclusion, and the S@H dolts refused the help.
There are some who would cheat, and there are others who want to show how 'their' processor excels. You can't drop someone in either group without reviewing their code.
-ck
-- This sig is only a test. If this were a real sig it would say something witty. --
But with any experiment, you must be sure that ALL conditions are the same everytime it is repeated, to be sure the results are as accurate as possible.
"Oh, the reason you couldn't reproduce my experiment is that your conditions weren't the same. Jupiter was in Taurus when you did it."
Science isn't about making ALL conditions the same. Science is about deciding what conditions are important, and making them the same. (In that regard, your comments are very scientific: you've decided what's unimportant and actually completely forgotten about it.) So one of you is arguing "implementation is unimportant, algorithm is what matters", and the other one is arguing "who writes the implementation really is important, because there's a finite chance they'll screw it up, by mistake or on purpose". Guess what: you're both right.
So find a solution that solves both problems. Here's a hint: it's going to involve social engineering. The scientists and the hackers are going to have to actually listen to each other if they want to reach a good accomodation. It's not just a matter of the hackers shutting up and doing what they're told.
Preferential Voting: easy as 1-2-3