Slashdot Mirror
Hosting Problems For distributed.net
Yoda2 writes "I've always found the distributed.net client to be a scientific, practical use for my spare CPU cycles. Unfortunately, it looks like they lost their hosting and need some help. The complete story is available on their main page but I've included a snippet with their needs below:
'Our typical bandwidth usage is 3Mb/s, and reliable uptime is of course essential.
Please e-mail dbaker@distributed.net if you think you may be able to help us in this area.'
As they are already having hosting problems, I hate to /. them, but their site is copyrighted so I didn't copy the entire story.
Please help if you can." Before there was SETI@Home, Distributed.net was around - hopefully you can still join the team.
Could they just move the project over to SourceForge?
Jouster
Maybe they should go in for distributed hosting, like say one machine that just houses the IP address and a few thousand mirrors that the requests can be directed to as they come in. Not only is it a project that is just ASKING to be performed by distributed.net, but if they make some catchy point and click (i.e. EASY to use) clients that anyone with a large following can use, we might see the end of such things as Slashdot subscriptions and a resurgence of the "community" feel of the web.
"Your superior intellect is no match for our puny weapons!"
You've now got 10,000 readers hovering over the link, "Ooh, should I, shouldn't I?", then thinking f**k it and clicking anyway.
/.'ing ;o)
A slow, painful, prolonged,
The RC5-64 challenge is currently at 73%, moving fast. Can you imagine the project shutting down just now?
There are numerous things you just couldn't "distribute." The keys have to be served from somewhere, they must be tracked in real-time from somewhere, and they must be accepted/processed somewhere. Stats must be compiled and then put into a single database. To distribute this to multiple computers would cause the amount of bandwidth used to rise to an extreme level, far beyond what it is now. (ie. send out the info, let each node process it, receive the data from each node, hope to Christ it's right)
:)
Next, the integrity of the project gets called into question the moment you begin allowing clients to check processed blocks. The number of fals positives could easily shoot through the roof. Also, a computer with bad memory or simply running a faulty OS (such as Win9x/ME) could overlook a true positive, thereby virtually obliterating the project (ie. "we're at 100% completion with no result, guess we start over?")
As stated above, stats would be impossible to do in this manner, and the same applies for key distrobution. One could argue that the total keys be distributed amoung thousands of nodes and handed out from there, but you create more problems then you solve. You still need a centralized management location to keep track of keys that have or have not been tested. Imagine a node going offline permanently or simply losing the keys it was handed. Suddenly, a large block of keys is missing. As it stands now, the keymaster simply re-issues the keys to someone else after a couple of weeks of no response from the client it sent the original blocks to. Under a distributed format, the keymaster would have to keep track of which keys went to which key distributor, which of those came back, which of those need to be redistributed, where they... (you get the message.)
Next you run into another problem of integrity. What's to stop each distributed keymaster from claiming it's own client is the one that completed all blocks submitted to it. Consider this example, central keymaster sends out 200,000 blocks of keys to keymaster node 101. Keymaster node 101 distributes these keys to a bunch of clients which process the blocks, then send them back to keymaster node 101. Keymaster node 101, which has been modded slightly, then modifies each data block, changing the user id to that of the keymaster's owner, thereby making it appear that any block coming back from keymaster 101 was processed by keymaster 101. It might be easy to spot, but then how to you find out who to give credit to?
The webpage doesn't attract the majority of the bandwidth; the projects do. Distributing the projects would be disasterous, as many have already tried taking advantage of the current system to increase their block yields through modded clients. Luckily, this is easy to spot for now. Under a distributed system, this would be next to impossible. All this, and I've yet to make mention of the fact that the code would have to be completely re-written to work alongside a custom P2P application, which would add months of development to a project that probably only has weeks or months left in it.
In short, someone host the damn thing, k?
-- "Government is the great fiction through which everybody endeavors to live at the expense of everybody else."
For some time the only one around was seti@home which analyzes noise from space, I think, in search for alien lifeforms, then there's distributed.net doing crypto and math stuff, (correct me if I'm wrong). And then there's people like Intel running medical research in areas like cancer and alzheimer.
I don't know about you, but to me medical research feels a somewhat more beneficial to humanity than search for aliens. Don't get me wrong, I'm not saying that the work done by seti and distributed isn't important or shouldn't be done, just that there's other research that might be more worthwhile supporting.
That's just my opinion, but if you feel the same way, checkout this site.
Seti@home searchs a fairly insignificant portion of the sky for a completely insignificant number of signals with an un-optimized application which does little more than make pretty color pictures on the screen.
Cancer research? I've yet to see a viable distributed project for cancer research. By that, I mean an organized effort with real data, a complete and concise goal, and a clean method for reaching that goal. Distributed raytracing? More pretty pictures on the computer screen.
You want to draw pretty pictures, I want to brute force an encrypted message to prove current laws regarding encryption are draconian and need to be changed immediately. Gee, I can't imagine why anyone would think dnet is more usefull than raytracing....
-- "Government is the great fiction through which everybody endeavors to live at the expense of everybody else."
A continuous three Megabits per second works out to somewhere just under a Terabyte a month. Not going to be cheap.
10 PRINT CHR$(205.5+RND(1)); : GOTO 10
There's absolutely no evidence to suggest that the government will ever change crypto laws based on what happens at distributed.net.
Its not like those in government who are responsible for consulting in these matters (NSA, etc) aren't aware of the issues at play here with current export-level encryption -- if you think that they are somehow unaware of these issues and dnet is required to bring them to light, please pass the crack pipe.
...by VAPORIZING us!! YEEAAARRGGHH!!!
Because if a few thousand unspecialized computers can brute force the best encryption allowed by law with minimal optimization and research, then we have some good reasons to push for the law to be changed.
That might have been true when d.net was working on DES, but things have changed.
I think a more accurate wording would be
Because if a over ten thousand computers, working for three years, can't brute force 64-bit encryption, when 256-bit encryption is readily available then we have very little reason to push for the law to be changed.
Tarsnap: Online backups for the truly paranoid
"Cancer research? I've yet to see a viable distributed project for cancer research. By that, I mean an organized effort with real data, a complete and concise goal, and a clean method for reaching that goal. "
http://members.ud.com/home.htm
This is real research, worked on by United Devices, helped by the University of Oxford, Intel and the National Foundation for Cancer Research.
It meets all your criteria- this is from their site:
"The research centers on proteins that have been determined to be a possible target for cancer therapy. Through a process called "virtual screening", special analysis software will identify molecules that interact with these proteins, and will determine which of the molecular candidates has a high likelihood of being developed into a drug. The process is similar to finding the right key to open a special lock--by looking at millions upon millions of molecular keys."
graspee
I get about 2.0 Mbps from my "2.2 Mbps" SDSL connection. If two other folks such as me were to pitch in, I think we could handle it. Not sure if this would classify as business use, if so I would have to hand over another $25/month to my ISP.
I, for one, have boxen and bandwidth to pull off 3 Mb/s of CPU-intensive network traffic 24/7
Sweet! What sort of connection is that? The cable modem provider in my area offers very limited "business" symmetric connections up to 5 Mbps, but they charge dearly for it. A lot cheaper than a fractional T3, though.
The "keymaster" (the machine that utilizes the ~3Mbit/sec) already distributes larger regions of uncomputed work to all of the "fullservers", which are the ones that in turn distribute the actual work to clients after splitting the blocks into sizes that correspond to what is needed by clients. You can see the list of all of the fullservers at http://n0cgi.distributed.net/rc5-proxyinfo.html
All of the chatty, multi-step network communications overhead with dealing directly with the clients is done at the fullserver level, including doing a windowed-history based coalescing on result submissions.
Don't waste those cycles! Put them to use! http://www.distributed.net/
Because distributed.net is a purely volunteer project, many of its staff also have their paid day-time jobs working for United Devices (who are responsible for the THINK Cancer project). That includes myself, Nugget, Decibel, Moose, Moonwick
Don't waste those cycles! Put them to use! http://www.distributed.net/
Distributed.net has gotten to be a more or less pointless project by now.
Originally, the point they wanted to make was that 64-bit RC5 was not strong enough to protect privacy.
They started, what, 4-5 years ago? About 30 000 computers running for 4 years can't break 64-bit encryption. Geez, I'd say that, if anything, the conclusion would be that 64-bits is plenty for shopping etc. unless you've got some really _big_ secrets. Certainly plenty for day-to-day mail. More or less the opposite of what they wanted to prove.
Nowadays they've added the OGR stuff to appear at least a bit more usefull, but in reality, the applications of those results are very limited.
Really, the right thing to do is not to waste power on such pointless projects.
--
GCP (Moderation suggestion: -1 Disagree)
Client downloads are PGP signed http://http.distributed.net/pub/dcti/v2.8015/ and are served by machines that mirror it (via rsync over ssh) from a tightly controlled host, which is not one of the servers that actually publicly serve the files. Although the binaries are pre-compiled, the original source code is open for review at http://www.distributed.net/source/
Don't waste those cycles! Put them to use! http://www.distributed.net/
I was interested in what you said about Intel & their distributed cancer research, so I checked it out. Unfortunately, their site is a little scarce on the details of who this research benefits.
However it does mention that finding drugs to combat various diseases is a first priority. So I assume that a particular pharmecutical company would benefit from this, as would a small percentage of people with cancer who also have private health insurance.
I would want my CPU time going in open-source medicine, and not someone else's patent that will be abused to make the most money possible.
I'm not saying that this is the case with Intel's distributed cancer-curing client, but it kinda looks like that given the lack of details of beneficiaries.
Anyone know for sure?
I might email them...
That figure is actually closer to the current average peak. We in fact currently have an ipfw bandwidth limit on the machine to limit it to 3Mbit/sec and it mostly stays under it. We just over-quoted that figure a little bit in our announcement so that there would be fewer concerns over some marginal potential growth and try to factor in some of the bandwidth peaks.
Don't waste those cycles! Put them to use! http://www.distributed.net/
I just saw this statement at the bottom of their front page:
distributed.net and United Devices join forces: distributed.net and United Devices have announced a partnership which will combine the skills and experience of distributed.net with the commercial backing of United Devices. Several distributed.net volunteers are leaving their old day jobs and joining United Devices full time. United Devices will be providing distributed.net with new hardware and hosting services, as well as sponsoring a donation program that will help support distributed.net's charitable activities."
I guess they are okay for the time being?
Co-founder and designer at Music Nearby: http://musicnearby.com
Getting donations of bandwidth and hosting is harder because those are ongoing commitments (including potential staff-support, and physical colo access, etc).
Direct money donations are also somewhat hard to get. Fortunately distributed.net is a 501(c)(3) organization, which means anyone can donate and receive an income tax writeoff (see articles of incorporation). Tax day is coming up soon, folks! :)
Don't waste those cycles! Put them to use! http://www.distributed.net/
However, this doesn't reduce the bandwidth at the keymaster any further, since this sort of splitting is already also being done at a larger scale between the keymaster and fullservers (and the bandwidth issue is with the keymaster, not the fullservers).
Don't waste those cycles! Put them to use! http://www.distributed.net/
Since it's a "contest" with cash prizes, why not charge people to enter.
d.net can't do that because under the regulations of many U.S. states, that would be considered "gambling."
Will I retire or break 10K?
Thier site is popular enoug that it would seem to be a good time to experiment with moving the http stuff to freenet, since it's only updated once per day. The people willing to download the dnet client are would seem to be some of the most willing people to download the freenet client. Freenet is designed so that the slashdot effect actually increases reliability and speed of acess for the commonly requested data. Distributed.net would seem to have reached a critical mass of readership in order to have reasonable reliability for its freenet page. Your could have the client get your team and individual scores sent to it as part of the block submission cinfirmation.
It would seem to me that they could arbitrarily reduce their bandwidth requirements by increasing the minimum size of keyspace portions they're handing out. It would seem that thier project traffic would be (or could be made) the same for each work unit, regardless of the size of the work units. Bigger work units are really only a problem for clients that are turned off and on regularly. They client still only needs to keep track of current state (current key in the case of RC5), the final state of the work unit (last key to check for RC5) and the current checksum for the work unit. None of these change in memory requirements as you increase work unit sizes. 99% of the people don't know the work unit size anyway, so changing the work unit size won't cause many people to complain, particularly if it's necessary to keep dnet hosted.
Unless I'm mistaken, the server really only needs to send the client a brief prefix identifying the message as a work unit, followed by "start" and "stop" points for the computation. For RC5, this would mean a 64-bit starting key and a 64-bit ending key. I haven't sat down and worked out the cannocalization scheme for GRs, but it seems that they are countable (in the combinatorics sense, not the kindergarten sense) and could be represented fairly compactly. The current minimum ruler length need not be sent, snce you'd probably always want the client to send back the minimum ruler length in it' work unit anyway. The client would need to send back a work unit identifier (this could be left out, but it's not strictly safe) and an MD5 sum of all of the computational results or some other way to compare results when they duplicate work units. (A certain percentage of the work units are actually sent tomultiple clients in order to check that everyone is playing fairly.)
Copyright Violation:"theft, piracy"::Anti-Trust Violation:"thermonuclear price terrorism"<-Overly dramatic language.
It does that with a ton of people until it finds the right key. It will eventually crack every crypto they throw at it, because it's only a matter of time.
Seti@home is searching for something that they don't even know if it's out there, and can you imagine the impact if they do find PROOF that there's life somewhere else? That's far more important then stupid crypto keys and such
the UD cancer treatment, while iffy because it's probably set up to benifit a company still has a HUGE impact on EVERYONE'S life.. I don't know anybody that either hasn't had cancer or a family member that has had cancer, and to find a cure!
Free Mac Mini
By proving that RC5-56 can be broken by simple home PCs (with an algorithm as simple as you call it "counting to a million by ones", they IMHO did a large part to educate lawmakers that the age old U.S. export restrictions have to be overturned.
And they succeeded in this.
What I however don't understand is why they kept doing their cryptography projects afterwards. Proving that RC5-64 is breakable while you can buy 256 bit encryption freely is indeed just a stupid waste of CPU cycles and bandwidth.
I'd like to see them discontinue RC5-64, and concentrate their work on OGR and maybe on other, new projects.
Honestly.
We all know that eventually, the key is going to be found, and some stupid message will be deciphered ("Congratulations on solving the 64 bit challenge. blablabla")
Why waste trillions of CPU cycles and thousands of $ in bandwidth to find something out that we already know is true?
Because they started RC5-64 over four years ago and probably didn't change their frame of reference since then, only the multiplier.
Sort of like how some PC magazines do benchmarks of things such as hard drives with old systems, to ensure that you can compare last week's results with some numbers published two years ago, in a semi-meaningful way since the only thing changed is the different hard drive.
Esli epei etot cumprenan, shris soa Sfaha.
> will determine which of the molecular
> candidates has a high likelihood of being
> developed into a drug
>
Which will then be sold back to you at prices, where dying from cancer is probably the better choice. Profits, amazingly, do not get donated to the Free Software Foundation but to lawyers fighting the demand for affordable generic drugs. The drug-empire CEO's meanwhile sip martini's floating in their yacht just a couple miles off the coast of the Karposi-Belt...
You know, I would help out with all this distributed computing stuff, but my spare CPU cycles are all taken up running multiple instances of Progress Quest.
The original idea of distributed.net dates back to when the government was conspiring to restrict the number of bits in encryption and students protested that 64 bits wasn't enough. Well it may be technically breakable but economics made it unbreakable in the end.
it sure does sound more impressive to a non-tech than 11000 PIII Xeon 800s or whatever the equivalent would be.
Big number in the front... ooohh look, shiny things.
Moderators: up the parent as Informative please.
I agree that the medical research might be more worthwhile to support, but AFAIK there are only Wintel clients available. (Case in point, United Devices and even your own link to Intel.)
That leaves an awful lot of non-intel boxes, and even non-windows intel boxes with spare cycles that can't participate. Until they have the option to do so, I anticipate a lot of cycles going to 'less worthy' causes...
If you find the key, you get $2000, I don't know about you, but that would sure improve MY daily life.
Please feel free to email sales@tiernetworking.com for more information.
I'm not really sure what to make of your comment. First, there's plenty of good connectivity in Austin, Houston, San Antonio, and Dallas. More importantly, we have a large concentration of staff in Austin, which is very important whenever physically working on the hardware is required.
I sent an e-mail to my guys at pair.net and they said they would look into it. They also said thanks for pointing the site out. Maybe some of you guys can try some other hosting sites? Worth a shot!
JOhn
Campaign for Liberty