Slashdot Mirror
Hosting Problems For distributed.net
Yoda2 writes "I've always found the distributed.net client to be a scientific, practical use for my spare CPU cycles. Unfortunately, it looks like they lost their hosting and need some help. The complete story is available on their main page but I've included a snippet with their needs below:
'Our typical bandwidth usage is 3Mb/s, and reliable uptime is of course essential.
Please e-mail dbaker@distributed.net if you think you may be able to help us in this area.'
As they are already having hosting problems, I hate to /. them, but their site is copyrighted so I didn't copy the entire story.
Please help if you can." Before there was SETI@Home, Distributed.net was around - hopefully you can still join the team.
Could they just move the project over to SourceForge?
Jouster
Maybe they should go in for distributed hosting, like say one machine that just houses the IP address and a few thousand mirrors that the requests can be directed to as they come in. Not only is it a project that is just ASKING to be performed by distributed.net, but if they make some catchy point and click (i.e. EASY to use) clients that anyone with a large following can use, we might see the end of such things as Slashdot subscriptions and a resurgence of the "community" feel of the web.
"Your superior intellect is no match for our puny weapons!"
You've now got 10,000 readers hovering over the link, "Ooh, should I, shouldn't I?", then thinking f**k it and clicking anyway.
/.'ing ;o)
A slow, painful, prolonged,
The RC5-64 challenge is currently at 73%, moving fast. Can you imagine the project shutting down just now?
No, they can't shut down yet! I have to break 10,000 in the rankings!
:(
Good Lord, what shall I do?
-- "Government is the great fiction through which everybody endeavors to live at the expense of everybody else."
Seriously, what is the current state of p2p-networking when serving common html-pages would be the thing to do?
The fact that such a big world-wide project is bound to be hosted near Austin shows that computing technology still has a long way to go...
Brain Tags |
What has it accomplished besides searching a keyspace with a known length and golumb rulers? Seti@home, cancer research, or that distributed raytracing screen saver is far more more useful.
An Education is the Font of All Liberty
we need your help!
URGENT: We have recently learned that our long-standing arrangement with Texas.Net (formerly Insync) would end at noon, Friday, March 22. Through an agreement with Insync, we were hosted at no charge for many years. Though we have tried to make other arrangements with them or to continue our current service until we can make other arrangements, in the end we had no choice but to move.
Several of the Austin cows made a road trip Friday morning to retrieve our equipment from their colocation facility.
We have no reason to complain about Texas.Net or their current decision. As a business, they chose to donate to us for a long time, and have now decided that they must stop. In dbaker's words in a letter to Texas.Net: "Our experience with Insync has been excellent; I've never been happier with an Internet provider. I've recommended them (and indirectly, Texas.Net) to everyone and even this [situation] won't change that."
Though United Devices has kindly offered to colocate our primary servers for a short time at no expense, we find ourselves in the market for a new ISP. If any of our participants work for a major ISP in Texas (preferably within a few hours of Austin, but we're not picky), and would be willing to donate colocation space and connectivity, we would eagerly like to speak with you. Our typical bandwidth usage is 3Mb/s, and reliable uptime is of course essential.
Please e-mail dbaker@distributed.net if you think you may be able to help us in this area.
Distributed net?
Correct me if I'm wrong, but isn't this the outfit which is concerned with breaking low-grade crypto? How's that going to improve my daily life? I'd much sooner donate my CPU cycles to the evil international pharmaceutic corps which does benefit cancer study. If you get a rash from commercial ventures, there's the folding@home. It's more like basic research, so it won't produce any miracle cures, but it might eventually lead to research that could.
But breaking crypto? Why?
I have always found distributed.net to be a relative structured organization. Their software with personal proxies made joining much easier than the Seti project, esp for people behind corporate firewalls. Small unobtrusive clients (esp for the des/rc5 projects) for a LOT of platforms.
It would be a shame to see them disappear. They've had/has a lot of cumulative computing power, and it ought to be put to real use.
Ah, the days of installing the res/rc5-42 clients on lots of 386 and 486 machines and actually having them do some real computing....
//TheToon
There are numerous things you just couldn't "distribute." The keys have to be served from somewhere, they must be tracked in real-time from somewhere, and they must be accepted/processed somewhere. Stats must be compiled and then put into a single database. To distribute this to multiple computers would cause the amount of bandwidth used to rise to an extreme level, far beyond what it is now. (ie. send out the info, let each node process it, receive the data from each node, hope to Christ it's right)
:)
Next, the integrity of the project gets called into question the moment you begin allowing clients to check processed blocks. The number of fals positives could easily shoot through the roof. Also, a computer with bad memory or simply running a faulty OS (such as Win9x/ME) could overlook a true positive, thereby virtually obliterating the project (ie. "we're at 100% completion with no result, guess we start over?")
As stated above, stats would be impossible to do in this manner, and the same applies for key distrobution. One could argue that the total keys be distributed amoung thousands of nodes and handed out from there, but you create more problems then you solve. You still need a centralized management location to keep track of keys that have or have not been tested. Imagine a node going offline permanently or simply losing the keys it was handed. Suddenly, a large block of keys is missing. As it stands now, the keymaster simply re-issues the keys to someone else after a couple of weeks of no response from the client it sent the original blocks to. Under a distributed format, the keymaster would have to keep track of which keys went to which key distributor, which of those came back, which of those need to be redistributed, where they... (you get the message.)
Next you run into another problem of integrity. What's to stop each distributed keymaster from claiming it's own client is the one that completed all blocks submitted to it. Consider this example, central keymaster sends out 200,000 blocks of keys to keymaster node 101. Keymaster node 101 distributes these keys to a bunch of clients which process the blocks, then send them back to keymaster node 101. Keymaster node 101, which has been modded slightly, then modifies each data block, changing the user id to that of the keymaster's owner, thereby making it appear that any block coming back from keymaster 101 was processed by keymaster 101. It might be easy to spot, but then how to you find out who to give credit to?
The webpage doesn't attract the majority of the bandwidth; the projects do. Distributing the projects would be disasterous, as many have already tried taking advantage of the current system to increase their block yields through modded clients. Luckily, this is easy to spot for now. Under a distributed system, this would be next to impossible. All this, and I've yet to make mention of the fact that the code would have to be completely re-written to work alongside a custom P2P application, which would add months of development to a project that probably only has weeks or months left in it.
In short, someone host the damn thing, k?
-- "Government is the great fiction through which everybody endeavors to live at the expense of everybody else."
I find this an excercise in futility; if the protocols used to transmit the data are not available to /.ers, we cannot suggest a scheme that would be meaningful. If the blocks are indexed, and all that's returned is an "index <X> complete" message, then a system of proxies sending message like "indexes 1217-1250 completed by my subnodes" to the main server once every hour makes sense. If, on the other hand, the bulk of the data is used to verify that processing actually occured, and that it occured with the official client (which I suspect is the case), we would need to know details of the data being passed back and forth in order to help.
I know that I, for one, have boxen and bandwidth to pull off 3 Mb/s of CPU-intensive network traffic 24/7, but I'm not about to devote my precious resources to something that I don't understand, especially when I haven't even had the chance to ascertain that a solution that utilized my donated resources was, in fact, the best one.
Jouster
For some time the only one around was seti@home which analyzes noise from space, I think, in search for alien lifeforms, then there's distributed.net doing crypto and math stuff, (correct me if I'm wrong). And then there's people like Intel running medical research in areas like cancer and alzheimer.
I don't know about you, but to me medical research feels a somewhat more beneficial to humanity than search for aliens. Don't get me wrong, I'm not saying that the work done by seti and distributed isn't important or shouldn't be done, just that there's other research that might be more worthwhile supporting.
That's just my opinion, but if you feel the same way, checkout this site.
A continuous three Megabits per second works out to somewhere just under a Terabyte a month. Not going to be cheap.
10 PRINT CHR$(205.5+RND(1)); : GOTO 10
I told that girl to don't rip off fragile experimental projects.
Afterall she's a lady and she would understand that's coward action to phreack on big security holes... or not. To abuse distributed.net because exist on there big security holes.
We cannot rely on crackers' compassion, distributed.net must find a way to turn clients' security into a better level.
...by VAPORIZING us!! YEEAAARRGGHH!!!
more than 160000 PII 266MHz computers
This is a present? For me?
Cool, but they should have presenting me with 10000 Athlons 2000+ plus 10000 Geforce4's and 10000 Game Tits XP instead. I mean, does anybody know why they use PII's 266 MHz as a reference?
Distributed.net is somewhat seemlike but different to clustering piped via NIC's.
Few processing power is required to handle the I/O between machines, data compression and validation compared to the cpu cycles those will be spend on "internal" processing. This is true for both boxed clusters and distributed.net.
Clustering management doesn't make an impact so the cpus will do all stages of processing "in-house", but due to a different architeture distributed.net needs power to handle "160000 PII 266 MHz".
Just think of how many cpu cycles will be wasted if they are forced to shut down... boggles the mind!
"If all the clients just did random blocks of keys you'd expect the key to be found equally as fast."
That would lead to a lot of blocks being processed more than once before the entire keyspace was exhausted, increasing the time required.
I get about 2.0 Mbps from my "2.2 Mbps" SDSL connection. If two other folks such as me were to pitch in, I think we could handle it. Not sure if this would classify as business use, if so I would have to hand over another $25/month to my ISP.
Perhaps but how many blocks reported to dnet are valid in the first place?
A real experiment would not hand out the same block to only one person. Heck check out UD they claim to send the same block to at least five people. And I agree its a good idea. That chances that 5 people are cheats is less than the prob of only 1.
Besides, there are teams of thousands on dnet. Why don't they organize their own cache? We already know quite a bit of the keyspace is invalid.
My point is there is really no need for dnet other than stats and glory. My single computer stands about as much chance of finding the key inside dnet or outside dnet.
Tom
Someday, I'll have a real sig.
Distributed.net has gotten to be a more or less pointless project by now.
Originally, the point they wanted to make was that 64-bit RC5 was not strong enough to protect privacy.
They started, what, 4-5 years ago? About 30 000 computers running for 4 years can't break 64-bit encryption. Geez, I'd say that, if anything, the conclusion would be that 64-bits is plenty for shopping etc. unless you've got some really _big_ secrets. Certainly plenty for day-to-day mail. More or less the opposite of what they wanted to prove.
Nowadays they've added the OGR stuff to appear at least a bit more usefull, but in reality, the applications of those results are very limited.
Really, the right thing to do is not to waste power on such pointless projects.
--
GCP (Moderation suggestion: -1 Disagree)
In most areas of the country, a single rack in a colo / exchange facility costs $ 1500 per month or less, and 3 Mbps would cost ~ $ 1200 per month. They didn't say how many racks they need, but at that bandwidth, my guess is no more than one or two.
So, they have been getting $ 3000 per month or more of free bandwidth and rack space.
IMHO, if their work is really important, they should be able to raise $ 36K per year from the crypto community.
It's just a suggestion, but wouldn't it make sense just to link to the Google mirror, rather than the site itself?
Of course, don't bother trying if Google hasn't had time to cache the site yet...
What's this Submit thingy do?
Client downloads are PGP signed http://http.distributed.net/pub/dcti/v2.8015/ and are served by machines that mirror it (via rsync over ssh) from a tightly controlled host, which is not one of the servers that actually publicly serve the files. Although the binaries are pre-compiled, the original source code is open for review at http://www.distributed.net/source/
Don't waste those cycles! Put them to use! http://www.distributed.net/
I was interested in what you said about Intel & their distributed cancer research, so I checked it out. Unfortunately, their site is a little scarce on the details of who this research benefits.
However it does mention that finding drugs to combat various diseases is a first priority. So I assume that a particular pharmecutical company would benefit from this, as would a small percentage of people with cancer who also have private health insurance.
I would want my CPU time going in open-source medicine, and not someone else's patent that will be abused to make the most money possible.
I'm not saying that this is the case with Intel's distributed cancer-curing client, but it kinda looks like that given the lack of details of beneficiaries.
Anyone know for sure?
I might email them...
Do people running their own keyservers for their teams help with the bandwidth at all? If they requested (required?) that each time over a certain size run their own keyserver, might it help?
The masses are the crack whores of religion.
I run the Folding @ Home client on Linux, and it runs quite well!
I prefer to use my spare cycles for Medical research.
http://folding.stanford.edu
well, they are testing only with 500 users atm or so... not really that much. When Google is going to be operation for everyone, I think distributed.net is done (at least the rc5-64 and the current ogr will be).
Hmmz... I really hope I can disable the option in the google toolbar when it's fully operational. I prefer distributed over the whatever-google-is-going-to-choose project.
That figure is actually closer to the current average peak. We in fact currently have an ipfw bandwidth limit on the machine to limit it to 3Mbit/sec and it mostly stays under it. We just over-quoted that figure a little bit in our announcement so that there would be fewer concerns over some marginal potential growth and try to factor in some of the bandwidth peaks.
Don't waste those cycles! Put them to use! http://www.distributed.net/
Also available for OS X.
(same URL)
Folding at Home
My single computer stands about as much chance of finding the key inside dnet or outside dnet.
'cept that you probably wouldn't write a search client. I know that you seem to be saying that they're nothing and you're something, but *they're actually doing something*.
I just saw this statement at the bottom of their front page:
distributed.net and United Devices join forces: distributed.net and United Devices have announced a partnership which will combine the skills and experience of distributed.net with the commercial backing of United Devices. Several distributed.net volunteers are leaving their old day jobs and joining United Devices full time. United Devices will be providing distributed.net with new hardware and hosting services, as well as sponsoring a donation program that will help support distributed.net's charitable activities."
I guess they are okay for the time being?
Co-founder and designer at Music Nearby: http://musicnearby.com
Getting donations of bandwidth and hosting is harder because those are ongoing commitments (including potential staff-support, and physical colo access, etc).
Direct money donations are also somewhat hard to get. Fortunately distributed.net is a 501(c)(3) organization, which means anyone can donate and receive an income tax writeoff (see articles of incorporation). Tax day is coming up soon, folks! :)
Don't waste those cycles! Put them to use! http://www.distributed.net/
They use false positives to check for this. They send the client a deliberate false positive, that way since a cheater is going to be sending back all negatives, they can find the cheaters.
I've had enough abrasive sigs. Kittens are cute and fuzzy.
Since the distributed.net projects are very research oriented, it seems like a well connected university could provide the hosting and if the university was already into encryption research, the distributed.net projects would mesh quite well.
Since it's a "contest" with cash prizes, why not charge people to enter.
d.net can't do that because under the regulations of many U.S. states, that would be considered "gambling."
Will I retire or break 10K?
Internet connectivity in Japan (home of Imode
among other things) is way ahead of the states/
europe, so why not move the server there..
NTT sells business connections via fiber to
the internet (10mbps and 100mbps/sec), for
something like around $200 or $300 month
depending which you choose, including
the ISP fee. Connectivity to the US and Europe
is pretty good (low ping times), generally.
[BTW, You can get residential (ie. dynamic IP)
for about $75/month, which is what I have here in
my house..
..its very cool, the hardest thing was actually
finding a router that could route that fast,
(linksys..etc, are too slow, max out at 5mbps or
so), but some Japanese companies make them
now.. it connects to an 100mhz PC I got for
free from the bin, which now runs as a linux
webserver (and linux, being so cpu efficient,
has no trouble keeping up..as well as more modern
machines I use for my desktop), but the server
does a great job serving web pages & the like,
running video conferences to the US..etc..
So if you need bandwidth, come here..
Very cool..
Thier site is popular enoug that it would seem to be a good time to experiment with moving the http stuff to freenet, since it's only updated once per day. The people willing to download the dnet client are would seem to be some of the most willing people to download the freenet client. Freenet is designed so that the slashdot effect actually increases reliability and speed of acess for the commonly requested data. Distributed.net would seem to have reached a critical mass of readership in order to have reasonable reliability for its freenet page. Your could have the client get your team and individual scores sent to it as part of the block submission cinfirmation.
It would seem to me that they could arbitrarily reduce their bandwidth requirements by increasing the minimum size of keyspace portions they're handing out. It would seem that thier project traffic would be (or could be made) the same for each work unit, regardless of the size of the work units. Bigger work units are really only a problem for clients that are turned off and on regularly. They client still only needs to keep track of current state (current key in the case of RC5), the final state of the work unit (last key to check for RC5) and the current checksum for the work unit. None of these change in memory requirements as you increase work unit sizes. 99% of the people don't know the work unit size anyway, so changing the work unit size won't cause many people to complain, particularly if it's necessary to keep dnet hosted.
Unless I'm mistaken, the server really only needs to send the client a brief prefix identifying the message as a work unit, followed by "start" and "stop" points for the computation. For RC5, this would mean a 64-bit starting key and a 64-bit ending key. I haven't sat down and worked out the cannocalization scheme for GRs, but it seems that they are countable (in the combinatorics sense, not the kindergarten sense) and could be represented fairly compactly. The current minimum ruler length need not be sent, snce you'd probably always want the client to send back the minimum ruler length in it' work unit anyway. The client would need to send back a work unit identifier (this could be left out, but it's not strictly safe) and an MD5 sum of all of the computational results or some other way to compare results when they duplicate work units. (A certain percentage of the work units are actually sent tomultiple clients in order to check that everyone is playing fairly.)
Copyright Violation:"theft, piracy"::Anti-Trust Violation:"thermonuclear price terrorism"<-Overly dramatic language.
By proving that RC5-56 can be broken by simple home PCs (with an algorithm as simple as you call it "counting to a million by ones", they IMHO did a large part to educate lawmakers that the age old U.S. export restrictions have to be overturned.
And they succeeded in this.
What I however don't understand is why they kept doing their cryptography projects afterwards. Proving that RC5-64 is breakable while you can buy 256 bit encryption freely is indeed just a stupid waste of CPU cycles and bandwidth.
I'd like to see them discontinue RC5-64, and concentrate their work on OGR and maybe on other, new projects.
Honestly.
We all know that eventually, the key is going to be found, and some stupid message will be deciphered ("Congratulations on solving the 64 bit challenge. blablabla")
Why waste trillions of CPU cycles and thousands of $ in bandwidth to find something out that we already know is true?
You know, I would help out with all this distributed computing stuff, but my spare CPU cycles are all taken up running multiple instances of Progress Quest.
Seems to me like this story makes distributed.net obsolete.
The original idea of distributed.net dates back to when the government was conspiring to restrict the number of bits in encryption and students protested that 64 bits wasn't enough. Well it may be technically breakable but economics made it unbreakable in the end.
Moderators: up the parent as Informative please.
I agree that the medical research might be more worthwhile to support, but AFAIK there are only Wintel clients available. (Case in point, United Devices and even your own link to Intel.)
That leaves an awful lot of non-intel boxes, and even non-windows intel boxes with spare cycles that can't participate. Until they have the option to do so, I anticipate a lot of cycles going to 'less worthy' causes...
Because d.net uses a 100% non-random criterion to select a winner (namely, whether they found The Key), it's not a contest, it's a competition.
What's so "100% non-random" about how RSA Labs selected the key?
Will I retire or break 10K?
I probably wouldn't because what does this prove?
Nobody in their right mind uses a 64-bit key for a symmetric cipher anymore. Its essentially assumed that its not a good idea. Most people just default to a 128-bit key and pray that the rest of the cryptosystem is worth the same.
See what the dnet people miss is that while a long key is a good idea to encourage, good crypto practices such as
1. Lots of eyes on the code
2. Open peer review [encourages good coding lest you want to get embarrased]
3. Good passwords
4. Secure clients [e.g. no viruses]
etc...
are more important. All dnet is proving is that >250000 computers can attack a 64-bit key in four years. What they fail to mention is to attack a 128-bit key will take 2^64 more time [or more computers in the same time] to solve.
Whoopy.
Tom
Someday, I'll have a real sig.
Each project has it's own benefits. I completed 5000 SETI units and now I am looking for prime numbers. If you feel that strongly about medical research, then good for you. I did not like the bandwidth problems SETI kept running into. I decided my spare cpu cycles would be better spent elsewhere. I share the same concern others have expressed about how the medical research data will be used. Some companies think they can patent my genes :(
I recommend that people look at all of the distributed projects. I suggest that you can support more than one. We can learn from all of them.
BTW, the EFF supports GIMPS. Maybe I will get back that money I have been donating for the last few years ;-)
Yes, it's true.
:o)
The Knights of Xenu are currently ranked 45th...
They're team 3504, so if you aren't yet affiliated with the team, or are new to the distributed.net effort, please feel free to join that team and help bring more exposure to the insanity that is Scientology
Wouldn't the hosting problem be fairly minimal if they distributed the key generation/distribution/whatever using peer2peer tech? It's not as though it doesn't have the infrastructure there yet. People have already been willing to give up their cpu time, and I suspect that 10-25% would be more than willing to give up a small chunk of their bandwidth as well.
Google
Why do care? Why are so many organizations begging for money, bandwidth, or something else on /.? This is pathetic!!
I think the use of spare fart cycles is an excellent way to support science, but...
:(
;-)
Each project has it's own benefits. I completed 5000 [www.farts.com] fart units and now I am looking for prime farts [www.farts.net]. If you feel that strongly about medical fart research, then good for you. I did not like the bandwidth problems [www.fartfarm.com] kept running into. I decided my spare fart cycles would be better spent elsewhere. I share the same concern others have expressed about how the medical fart research data will be used. Some companies think they can patent my farts
I recommend that people look at all of the distributed farting projects. I suggest that you can support more than one. We can learn from all of them.
BTW, Breaking Wind: Legendary Farts [http://www.pitt.edu/~dash/fart.html] supports Literacy. Maybe I will get back those beans I have been donating for the last few years
"Face it, a nation that maintains a 72% approval rating on George W. Bush is a nation with a very loose grip on reality.
Please feel free to email sales@tiernetworking.com for more information.
3Mbps is puny and **could** not cost a thing network-wise in the right place given the right schedule. Most bandwidth is priced on the 95th percentile of usage, and is based on the highest of the inbound and outbound traffic. Here are a few things to note:
a) a large company or website with localtime 9-5 employee or website traffic greater than 10Mbps peak could do almost anything any time between 5-9. Limit the upload and download client activity to this timeframe and you have it solved for 2/3rds the day. Not a 100% effective, but not too significant if the workload pieces are multi-day. If you're concerned, you could somehow chain two of these companies around the globe and you have free bandwidth 24hrs a day.
b) a mainly outbound company (e.g.- large website) could receive data at any time of the day.
c) a mainly in-bound (corporate) company could send data at any time of the day.
I sent an e-mail to my guys at pair.net and they said they would look into it. They also said thanks for pointing the site out. Maybe some of you guys can try some other hosting sites? Worth a shot!
JOhn
Campaign for Liberty
I am sure that there are mechanisms in place to verify the PGP signature at regular intervals. All it would take is to run a friggen cron job to download the binary, PGP test it and return a result. Hell, i'll do it on one of my boxen at home, i'm sure the momentary interuption won't inconvenience the machine's real work of rc5 cracking. :)
"I'm tired of all this 'Aren't humanity great' bullshit. We're a virus with shoes" - Bill Hicks
That was quite a tangent ;-)