distributed.net Contest Setback
meisenst writes "distributed.net is reporting that there was a problem earlier on in the CSC project (a few weeks ago) and that about 25% of the blocks will have to be re-done. Their announcement is here." We've gotten more than a few submissions about distributed.net showing more than 100% of all packets processed, but held off posting it until we had the official word. And here it is.
Well, the bad part is that the CSC project was also affected. As most people know, the CSC project was supposed to be a very short key cracking project. The RC5-64 project isn't affected as bad since it is a long-time project.
Does anyone know what specifically caused the problems in the key server?
This is really annoying... Not the fact that we have to 25% again, but the fact that it took so long for them to say anything at all, while, as far as I can tell from the article, the problem was already known before we reached the 100% mark?
I will still continue to support them, however.
Cheers!
Costyn.
The Official Steve Ballmer Webpage
distributed.net is made up of humans, people make mistakes. It's not the end of the world, but it is a little annoying.
On the plus side, with the current key rates the remaining 25% will be cracked within a week or two.
The other plus is that the key may not be at the very end of the remaining 25% of the keyspace and therefore we will finish even quicker!
In Soviet Russia...michael would be rotting in Siberia!
Its only about a week of additional effort... probably the most hit will be the RC-64 project, which has been diminishing its rate slightly recently because of people flocking to CSC... hopefully, just another week is needed before getting all the people working back on RC-64.
Go distributed!
It's VERY important to note that "Official" work about the CSC keyspace being over 100% was ALREADY explained!
It works like this: D.Net rechecks blocks from suspicious clients. D.Net rechecks they by handing out those blocks again. D.Net has said repeatedly that they give full credit for EVERY block handed out in this way from the keyservers. AS SUCH it's easy to have the total go over 100% - this had NOTHING to do with the server problem. The article posted does not make this clear, and should be revised!
Hey Rob, howabout that tarball!
Oops... Another 24 hours now...
"Going to war without France is like going deer hunting without your accordion." - Jed Babbin
I wonder how many barrels of oil that wasted.
-- Virtual Windows Project
Something was mixed up in the article. The more than 100% of the keyspace checked was due to some reissueing some blocks on purpose to avoid people submitting fake blocks and such and get up their keyrate. So without the error dnet made they would get over 100% anyway.
Also RC5 was not affected because it is a longterm project, but because the error was somewhere in the CSC scripts.
And apparently they only discovered the bug just yesterday orso and not some time ago.
There is actually more than one problem with CSC, which is causing it to go over 100%
.plan, stating that 9-12% of the keyspace was being duplicated.
First: dbaker (Daniel Baker), released an official anno uncement explaining that the same blocks were being issued to multiple clients, to attempt to detect cheaters.
Then dbaker released another anno uncement in his
Second: nugget (David McNett), released an announcement stating that there had been a problem with the keymaster generating invalid blocks, resulting in 25% of the keyspace being duplicated.
So, one remaining question is, are they still sending out ~10% 'verification blocks'? Or have they abandoned that to allow us to complete the project faster?
We have reached 112% due to verification blocks and could reach 140% due to 25% of the keyspace being corrupt. However, if 12% of the 25% new blocks are duplicated, then we could reach about 155%...
--
David Taylor
davidt-sd@xfiles.nildram.spam.co.uk
[To e-mail me: s/\.spam//]
The invalid keyblocks have already begun redistribution, so yes they will be reprocessed (correctly this time). The person who correctly resubmits a validly re-computed version of that block will also receive credit. It's not possible for the old previously distributed version of the block to capture the second-pass credit for the block because the invalidly computed blocks are now being screened and discarded as they arrive at the keymaster.
Don't waste those cycles! Put them to use! http://www.distributed.net/
While most of dnet's client software is open source, the keymaster and proxy software is not. The problem that was found was found in the closed sourced keymaster software. This bug may have been fixed sooner if the source were available. I don't understand why the source needs to be closed for the keymaster software. Proxy and networking, I understand the arguments for (though I do not necessarily agree with).
TomG
While the main article does do a good job of saying that we as D.net'ers have to re-do about 25% of the keyspace, it does not cover the ending editors comment as to why we reached 100%. The real reason that D.net reached 100% is that we re-test somewhere around 10% of the blocks. This can be found in one of the portions of This Document, dBaker's plan. I hope this clears up any confusion as the two topics are related, but not the same.
J. Marvin
Participants will still receive full credit for their completed blocks, regardless of their validity. In the near future we will be supplying the stats server with the information it needs to properly discount the effects of any reverification work in the reported percentage.
--
This problem probably would have been caught sooner had the keymaster source code been released. How about it, d.net guys?
Speak truth to power.
... to nugget. In all honesty, it can take a lot to admit that you fucked up, and that's what this is. As he said; they're human too, and mistakes happen. We should just be glad that this got caught before we were at like 200% and wondering what the hell was going on :)
Now let's get cracking and finish this thing, and my applause again to nugget for getting it all fixed and admitting it.
---
Tim Wilde
Gimme 42 daemons!
Something seriously wrong indeed.
Seeing this note about the block corruption in CSC client, I checked my client.
I noticed that the new client had core-dumped on
my Linux machine this week, other programs are ok.
-- implying some corruption of major kind.
Has anyone else encountered this behavior ?
My client version is v2.8003-448 client for Linux
My system is SuSE 6.3 (i.e. libc6).
I run 2.3.x kernels -- but they haven't crashed ever --
so methinks this has is a problem with the
client binary.
yes, i have enabled csc and rc5 on client.
-ak
> We've seen this before, actually, and it's still in the process of being dealt with; the release of the Quake source to the hungry open source world.
I know this is slightly offtopic, but a week ago when I read Jeff "Bovine Lawson's .plan of Jan 6 and then his treatise on operational code authentication, I thought I was reading a Slashdot thread on how to correct the problem of cheating in an open-source Quake.
It seems that both distributed key cracking and distributed Quake playing :) face many of the same cheating issues. It's clear Jeff and co. have thought a lot about what problems would have to be "solved" before they can go open source (which they hope to in the future). I highly recommend that anyone interested in the whole Quake cheating problem read Jeff's thoughts first. (He even mentions why Netrek-style blessed binaries won't work in general.)
Graham "Teach" Mitchell, computer science teacher, Leander HS
I used to have 4 different email accounts of mine cracking keys for them on boxes at home, school, work, and I even tried to recruit friends to help me work on it. We had a decent team, and we were doing several thousand keys a day. It was fun.
But we've had problems with bugs in the clients, (like this extremely annoying transparency bug on windows) also the fact that there isn't a way to control your team efficiently - and to top it all off, they recently had a problem over there that DROPPED ALL OF MY MEMBERS FROM MY TEAM.
That was the last straw. That's a serious problem since 2 of the email addresses on my team aren't valid anymore so we can't rejoin that email address to the team. The boxes that are running using that email address are in a location we can't go to to change the setting there.
So, after all of the hassle, it stopped being fun. Particlarly when they dumped so many people from their teams. I stopped participating, and distributed.net lost about 20 clients. Maybe I'll try SETI or some other CPU donation project that has their shit together.
-- Truth goes out the door when rumor comes innuendo. -- Groucho Marx
I understood from reading the d.net documentation that there is an issue of blocks that were never returned to the keyserver. These lost blocks were counted in the percentage but once the project "complete", the non-returned blocks would have to be redistributed assuming that the key had not already been found.
That could account for at least part of the 25%. Especially since the Windows client tends to lose the packet that it is working on when the machine crashes.
Sure, if you do not feel that it is fun any more take your ball and go home. But no that you have had your little fit, maybe you want to act like a grown-up for a little while and take a closer look at the situation.
It is not distributed.net's problem that you cannot access your systems to update information that is no longer valid. Yes they have had some issues as of late, but they have made a real effort to correct them, maybe you should send an e-mail and demand your money back. Oh, that's right you didn't send them any money.
Good luck finding ET.
all persons, living and dead, are purely coincidental. - Kurt Vonnegut
Yeah, I've been bitten by that. CSC for the Mac and Tru64 Unix clients took a while to arrive compared to the Wintel/Linux/etc CSC clients. Of course, dcypher only has clients for the x86 architecture on 3 OSes...guess they don't want my cycles.
As for seti@Home, they haven't exactly avoided similar problems either. Complaints about duplicated blocks and un-optimized client code for their project have been posted on /. before.
I was wondering if anyone had any insite into whether or not the same problem could crop up with RC5. I've (http://stats.distributed.net/rc5-64/psummary.php3 ?id=74973) been working on this more long term project for nearing 600 days, and a simalar mishap could be devistating to our time to completion.
Comment removed based on user account deletion
1) If you were trying to cheat, we've noticed you and your efforts are ineffectual.
2) If you somehow have a broken client, read-only disk, or have run out of disk space and are inadvertantly submitting duplicates, you might want to check on it. You might be wasting cpu or network bandwidth that you are not aware of.
Don't waste those cycles! Put them to use! http://www.distributed.net/
cute, but we have only made clients available for processors with 32-bit registers, which means 386 and higher. :)
Don't waste those cycles! Put them to use! http://www.distributed.net/
Comment removed based on user account deletion
In fact, the closer the project gets to 100% completion, the more quickly unreturned blocks are re-assigned (and re-re-assigned) just to try to get somebody (anybody!) to crunch them and return the results. When a project begins, a block of 10-day-old keys is still mostly useful to the project. But near the end, there's a very high chance most of those keys have been handled already.
Your computers have a finite amount of compute-power to offer the Distributed project. The best thing you can do with that power is make sure that 100% of the blocks you work on are useful to the project. The closest you can get to guaranteeing this is to process the newest blocks you can get from the keyserver.
"But what if the keys I throw away had the one!" Keys are assigned in random order from the keyserver. Nobody knows if the one is in the beginning, middle, or end of the entire sequence of keys to check. The winning key has an equal chance of being in the old block you're considering throwing away as it does in any new block you could download from the keyserver, so what the heck - just take the new keys. (In fact, there's a slightly higher chance that a new block will have the winning key - some parts of your old block may have already been processed by others, right? And nobody's won yet, right?)
And criticism from ACs isn't?