Cheating at Seti@home

Megor writes "Well it was bound to happen, people are cheating on Seti@home to inflate their work unit statistics, and the people who administer Seti are ignoring the complaints. ZDNET has an article explaining how they are cheating."

Comments

Interesting paper on this subject

[taviso]

Theres an interesting paper on this subject available here. well worth a read.

Motivation?

[Enocasiones]

What drives people to do this? You may brag about being first, but still, you'll be first together with all your teammates. Lots of people to share the credit, not much left for an individual. And the fact that the cheating could corrupt the results just makes matters worse.

Re:Motivation?

[Anonymous Coward]

I'm not an anonymous coward. My handle in seti is EG, I crunch for EXDC and am a member of ARS technica.

One of the cheats actually indexes the name of the wu. the name of the wu contains data that locates the data to a chunk of the data tape.

by indexing the same file name over and over to bypass the duplicate checking routines, you are introducing wu's that do not correlate to their proper location in space. Resubmit such a wu thousands of times with the name indexed each time it's sent destroys the baseline of the science database. Copy the same wu to 1,000 other people for submission multiplies the error a million fold. it is very conceivable that there might be whole chunks of wu's results that cannot be scientifically normed to ANY location on the starmap! It does effect the science data.

Re:Motivation?

[mosch]

A lot of people are assuming that these people cheated in order to up their stats (because seti stats get ya mad bitches!). I don't buy this for a second.

I believe that these "cheaters" are in fact alien lifeforms, who are covertly working to sabotage the fine research being done by the seti@home project. The project was coming close to discovering the location of their invasion force, so a crack team was sent to earth to disable the earth information gathering project, and to lay the blame squarely at the feet of other earthlings.

those aliens weren't quite clever enough though, were they...

Holey Moley!

[Anonymous Coward]

A first post which is

1. on-topic
2. gets modded up

What's this world coming to?

Wasn't cheating to be "impossible" ?

[holle2]

I thought the move to close the source of SETI@home back in the old days was meant to stop the cheating ?
Could it be that the protocol should be redesigned to contain, say digital signaures embedded into the binary (well not really a save place for that anyway ..)

Re:Wasn't cheating to be "impossible" ?

[ebbomega]

If anything, closing source opens you up to "cheats" because every time that an exploit/cheat comes up, you don't have OpenSource-support to fix it sooner rather than later.

Closing source isn't like sealing a tank. It's more like building a beaver-dam.

Re:Wasn't cheating to be "impossible" ?

[anshil]

Closing the source does not help a bit. After all you give a binary to your "foe", thats enough. Look in example to Ultima Online, they encrypt the stream already in 10 layers or so, with constant changing keys, algorithmns and so on. But it is still beeing hacked, simple as that, you've a binary of the client, you can view the algorithmn on assembler basis, thats enough "source" code to hack anything assuming enough motivation and time.

Look at all the companies trying to hinder people copying with copy protection CD's, tongels and all that. Does it help? No it's all just a new challange for the hacker folk.

Re:Wasn't cheating to be "impossible" ?

[bogado]

In fact this is a hard problem to solve, since you have to trust the computations made on the client side. Every security protocol must not trust the client! The solution in this case would be punishing the guilt.

My opinion is that each login should have a key, this key would sign all the packects received from this particular login. Now for every X packects received from any client, you resend one of them to another user, of there is a mismatch in the packet You then redo the calculation with your trusted code, and checkout witch is cheating and then ban this user.

Since cheating is for achieving higher pontuation, no one would like to be banned, since this would mean that one would have to restart their statisics. Groups could also be punished if one of their members is cheating making them also responsible for their components. This would help to police the network.

The key I proposed is for guaranty that a good user could not be sobotaged by other people sending packets in his name. Also one couls adopt a policy of sending more test packects to user with higher, more suspicious, rates of delivery.

Re:Wasn't cheating to be "impossible" ?

[ComputerSlicer23]

Okay, your right, I'd just gotten done reading another link from the posts, there is one from distributed.net that discussed all this, and actually was much more interesting then the original link. They talk about using a public key to sign data. Not that differs per individual, but even that is easy to abuse.

It's *MUCH* faster to generate bogus data, then it is to check it. I could generate thousands of work units in an hour. If I was a bad person, I'd sign up for a new name for every single work unit. Or only every 10-20 units. If I can turn them in faster then you can test and reject them, I'm winning. The DOS will actual work. Remember, they can't possibly afford to check 1 out of every million blocks of the data they are sending out, otherwise they wouldn't be doing it as a distributed computing project. I can DOS them if they attempt to run a "trusted" version of the binary locally. I'd whoop them really bad and generating bogus data. The attacker isn't going to play nice and put all their work units in under a single name if they are attempting to subvert the process.

I'm speaking from the perspective of an attacker who want's to subvert the process not rack up a big WU total. If I really wanted to rack up a huge work unit total, I'd take all my units under one name, and then submit them via 10 signed keys when they are done, so they all look like proper work units. Then they never check them locally, because they got verified by 10 different people as the same. How handy... If they have a set of "trusted" users that have to verify all the blocks, then all they should have the trusted people run the binaries, because everybody else will be throttled by the rate of the trusted ones.

In the end, they really can't check anything locally or only by trusted users, because locally and trusted users doesn't scale, that's why it's a distributed process. All the attacker has to do, is overrun the computing resources of the checker, and they win. It's not hard at all to pump to much data at them, because I don't need to do any real work to generate them, they have to do loads of work to check them. (Spoken like a man who use to grade tests.)...

Thanks,

Kirby

SETI Checking?

[rabiteman]

From the article:
One common technique used by cheats is to distribute partially completed work units to other team members' SETI@home accounts. One account is used to process a work unit until it is 99 per cent complete. It is then distributed hundreds of other team members who process the remaining portion of the unit and return it. The WU is credited to their accounts vastly inflating the quantity of public processing that appear to be dedicated to the project.

Let's assume cheating is going on, and is being perpetrated in this manner. Why doesn't SETI@Home check each WU as it's submitted and say "Gee, here's hundreds of people from the same team submitting the same WU with the same result within minutes of each other. Seems awful suspicious!"

Seems awful suspicious.

Re:SETI Checking?

[phragle]

Seti does check,

As I understand it, for each unitl they send a number of redundant units out and then compare the evetual results taking the most popular result to be the correct result for that unit.

Re:SETI Checking?

[rabiteman]

As I understand it, for each unitl they send a number of redundant units out and then compare the evetual results taking the most popular result to be the correct result for that unit.

Even if this is the case, the point remains that one group handing in 300 redundant copies of the same data processed the same way will skew the results. What if the guy who processed the first 99% had some kind of screwup along the way, and his team hands in 300 copies of his screwed-up data? The other 3 people who got the same WU and got the right answer will be 'outvoted' by Team Cheater, ruining the whole effort (for that particular chunk of raw data, at any rate).

Re:SETI Checking?

[phragle]

from the seti FAQ http://setifaq.org/faq.txt 2.4 I heard I was getting the same work unit as everyone else. Is the program wasting my time? Nope, because the only time you're giving it is time your computer would have wasted anyway. Yes, early in the program there were times when the same work units went out over and over, due to overloading of the SETI@home servers that were supposed to be making new ones to send out. (They didn't expect half a million people to sign up, and they don't have enough staff or computing power to keep up with it.) And since then, the same work units are still sent out to several people, for various reasons (for instance, more than half the people who signed up have never returned their work units, and probably dropped out) But new work units are being sent out too, so just leave your SETI@home program working and it'll take care of the details. Note: If workunits are sent out multiple times, they can be doublechecked by SETI@home.

Re:SETI Checking?

[Zocalo]

I don't know about Seti@Home, but if I were designing the WU submission / verification procedure, I'd be looking for anomalous submissions and verifying them on my own, trusted hardware. That includes all high positives and any units where you management DB is reporting a data validation mismatches - like more units returned than dispatched.

Ultimately the league tables are just a bit of fun to entice more people into getting involved, Seti@Home probably doesn't care about who leads the tables in the slightest, only about getting a result. As long as they are confident that a positive result will not slip by unnoticed, why get involved in an resource wasting arms race with the cheaters?

Re:SETI Checking?

[Coplan]

Simple solution...

Seti should track what it hands out (and I'm sure it probably does). In fact, it should probably track to who it sends it (again, it probably does).

If Seti sends out 30 WUs (abroad), it should know that if it gets 200 back, a flag should be sent up. If seti sends a WU to Bob, but not to Gregg, and Gregg sends THAT WU back, the one returned from Gregg should be voided.

This is not about preventing competition. Screw that...Seti shouldn't be concerned about this issue relative to that. Seti's concern should be plain and simple -- it should be protecting the integrity of the data. 'Nuff Said.

Ahem.

[acehole]

This has been going on for as long as seti@home has been running.

There are a variety of excuses people have for doing such things, such as :-

* making the program calculate units faster
* falsifying unit completion and results
* hoping they cheat enough so they can get up the top of a table

These people dont realise the problems with doing such things, If you contaminate the results with fraudulant and false data then you might as well forget about the whole project.

What happens if there really was something found, but due to the high rate of contamination that it was thought to be too good to be true and discarded. Consequences really need to be thought out before you start wrecking the hardwork of scientists and academics who are only doing a service for everyone's benefit.

Re:Ahem.

[taviso]

one of the major culprits for this was actually Microsoft, they had a scam going where they optimized the SETI software for Windows and then published the results to show how well their platform performed.

Theres more information in the setifaq, section 1.3.6

and on this usenet thread.

its a very interesting topic.

Re:Ahem.

[IIRCAFAIKIANAL]

What happens if there really was something found, but due to the high rate of contamination that it was thought to be too good to be true and discarded. Consequences really need to be thought out before you start wrecking the hardwork of scientists and academics who are only doing a service for everyone's benefit.

Not that I disagree with you overall, but if they thought they found something but the results were contaminated, they would just reprocess them. Now, what we should worry about is something being overlooked...

I might feel like a loser at times, but...

[ClassicG]

all I need to do to feel better about myself is to remember that there are people out there who are so bad that they need to cheat at Seti@Home in order to feel like something worth anything.

Our experiences from running the rc5-56 challenge

[jukal]

run at cyberian.org some 4 years ago was that people will do anything to get their team/name listed in the first page of the statistics. Some of the people were even arrested by police for hacking into machines to make them crunch rc5 for their name. And it seems this trend is only getting worse. This is kind of sad, because it is not very good for the reputation of such efforts.

it's been happening

[jokrswild]

as a participant of the Seti@Home project, this has been happening for some time. For those familiar, check out the stats for Overclockers.com Seti Team, of which i'm a member (insert a "Crunch for us!" flag here). We've suspected our number 1 memeber of cheating, but we have no proof. His numbers as of late were usually 0, until a few of our other memebers caught up. His Work Unit production started being upwards 300 or 400 a day.

People tend to loose sight of the fact that Seti@Home is for scientific purposes, and get caught up in the statisitics of it all. I'm in to the statisitics, so i'll load more computers with the Seti@Home client, not cheat.

Re:whyd they

[simong_oz]

2. Sell units on eBay (it does happen...)

Wow. The depths of humanity that eBay churns up will never stop amazing me.

The saddest thing of all is not that the units are being sold, but that there are people out there buying them. GET A LIFE

Ironcly

[isorox]

I'm researching seti for a final year comp sci project, and I've just handed in a draft about how its been secure, but how my distributed foobar will be open, and therfore more secure.

(dunno how to make it secure yet though)

Cheating is a big thing, as you can sell your work units on ebay!

500 units @ 25 euros
and http://cgi.ebay.co.uk/ws/eBayISAPI.dll?ViewItem&it em=2064169353 and
http://cgi.ebay.co.uk/ws/eBayISAPI.dll?ViewItem& it em=2064990327

what about peer review?

[caveat]

if i were one of the reviewers of this work for publication, and i even heard a whisper about cheating, i'd pack the whole pile of results up and ship them straight back as invalid.
if this is /serious/ scientific research, there should be absolutely zero tolerance for cheating, and any team even suspected of it should be summarily disqualified and have all their results tossed - not out of fairness for the competitors, but for the simple sake of scientific integrity...you can't have people cooking the books and then expect legitimate results.

One possible response...

[Alsee]

With the competition?s close just two months away

Seti@home may just sit back and silently allow these people to continue putting work into cheating, then at the close of the copmpetition throw out all bogus results.

Sort of getting revenge by letting them waste their time for another two months.

-

Why Cheat?

[orthogonal]

We're cheating because some 133t script-kiddie beamed a message to your solar system, some 50,000 of your years ago.

Only by perverting your Seti@home results will we prevent you from discovering our advanced, trans-lightspeed, galaxy-spanning civilization -- and from discovering that despite our accomplishments, our civilization will fall unless we conquer your planet for water/slaves/Kentucky-Fried-Human (please pick one).

Naturally, as an Alien Commader, I must gloatingly reveal our plans, with the excuse that you puny humans are too primitive to stop us even if you do know.

PS: That Orson Welles broadcast 64 years ago today wasn't a hoax. We got to him just in time.

I'm confused...

[Frnak]

Wasn't seti@home just about scoring high work units?

And what is this extraterrestial stuff people are going on about?

It's too competitive.

[pommaq]

Listen to the guy in the article:
"Basically, three years of work to get to the top of the teams and eight million WU later, it looks as though the top team is going to be beaten by cheating," said Nealon.

Sure, the stats are fun. But once you make a competition of it, people are going to start cheating - doesn't matter if the only reward is seeing your name at the top of your group in some brute force number-crunching exercise. Even the legal users care mostly about where they are in the stats, not about the point of the project itself. I mean, look at the popularity of ProgressQuest.
If I were SETI@Home, I'd remove the stats. Sure, you'd lose humungous amounts of CPU power when the $r1p7 kiddies abandon the project, but if you're getting millions of WUs of flawed data sent back to you, you need to do something drastic. Besides, they've got a pretty strong brand by now, and I'm sure a lot of users would stick with them just for the good of the cause, not just for the dubious honor of being at the top of the stats.

Simple solution

[wowbagger]

When the hand out the work unit, put a unit ID number on it, and sign it with a hash.

If they see the same ID being submitted by more than one system, zero the work unit totals for both machines.

BOOM! Cheating now carries a very high price.

Re:SETI will fail...

[Contact]

You're assuming a hypothetical "alien race" which operates on the same sort of timescales that we do. If they "live" a few orders of magnitude more slowly, then radio waves suddenly become (to them) a few orders of magnitude faster...

Bear in mind that even amongst cultures on earth, perceptions of timescales vary. I've heard the phrase "In Europe they think 100 miles is a long way; In America they think 100 years is a long time". Imposing human values on a hypothetical alien race is somewhat anthropomorphic...

Re:in an ironic way

[NanoGator]

"through this cheating the Seti@home project has proved that there is intelligent life in the universe... albeiet life with the ethics of the undead....

They definitely proved a deficiency of intelligence here on earth.

Official Word: not a problem for the science.

[SETIGuy]

Yet another overblow cheating report. Frankly, it doesn't really impact the science. The cheaters only process a small fraction of the total data and candidate identification doesn't rely on either a single result or a results from a single work unit.

Lets keep the the scope of the problem in perspective. What these guys are worried about is being in first place in the stats. I understand their concerns, but right now we have neither the funds nor the manpower to share them. Perhaps when SETI@home is shut down, and SETI@home II is running, we will go back and adjust the totals. Perhaps not.

SETI@home II will run under BOINC and will have more immunity to such exploits. The cost of such immunity will likely be a GUID for each machine running BOINC, in addition to a per user key pair. This, of course, will get slammed by privacy advocates. Hell, if Microsoft were doing it I'd slam them.

Right now our priorities are

• Keeping everything running.
• Identifying candidates for reobservation at Arecibo (sometime in the next 4 months or so).
• Building the SETI@home II data recorder.
• Getting Astropulse running as part of the BOINC beta.
• Finding enough funding to keep us running.
• Designing the SETI@home II analysis code.

Sorry, but fixing the stats can't be a priority right now. The extension of that to "SETI@home doesn't care about cheating" is extrapolating too far.

Re:SETI's a scam. Always was. So?

[Surt]

Ok, I know this is a troll but:

Care to venture how many tax dollars go to seti right now? I'll venture an impressive number: 0. Seti is not currently publically funded. That's right, its all voluntary donations.

I don't think anyone speaking for SETI has really made a claim such as: If we do X we will find life. Many times certainly they have said we _might_ find life.

The drake equation and its variables are not 'made up'. They are estimated as accurately as possible from star surveys of the galaxy. Only the last couple of variables are really open to any sort of scientific question, the others are pretty well settled. The question that should be asked is whether or not there are enough hospitable star systems out there that it might be possible for intelligent life to exist outside our solar system, and the answer is clearly yes.

Seti also has a clearly falsifiable premise: Seti claims that there might be other intelligent life inside of our galaxy (I believe that seti is willing to settle for our galaxy, since talking to anyone in a different galaxy is currently really beyond the realm of possibility for our technology). This claim is trivially falsifiable. Send a small probe to every start system, and survey any planets found. If no life is found, I believe that most if not all seti scientists will be glad to consider the question answered. Now sure, falsifying this way is a bit too difficult for us right now, so we're trying some other methods, but certainly within another century or so we should be well prepared to consider attempting a direct refutation of the seti premise.

Or you can view this from the other direction. Seti is a project engaged with the attempted refutation of the scientific premise that there is no intelligent life in our galaxy outside our solar system. In this case we might say that they are simply using the best tools available to them right now, and we can expect that within the next 10,000 years or so that the question will be pretty well settled by advances in our technology, but it might be settled much sooner than that, and the potential value of settling the question early is tremendous.

To clear up a last point, there are 2 questions I think seti would like to answer:

1) Is there life outside our solar system, inside our galaxy.

2) Is any of (1) that we discover, also intelligent.

I think both are clear and falsifiable.