Slashdot Mirror
Seti@Home Bandwidth Problems
reflexreaction writes: "With so many of the /. users actively using and supporting Seti@home, many of you have realized that in the last couple of weeks that Seti has had some serious problems receiving completed data and getting new data to process from its 3 million members because of network bandwidth problems. All the gritty details are here. The article details some things that users can do to alleviate some of the problems including connecting during off hours and downloading more than unit than once using programs like SetiQueue for PC and Seti Unit Manager for Mac. Donations are also accepted. There is also a plea for bandwidth donations. It will be truly unfortunate if this page becomes /.ted without benefit from /. users."
2/6/2002
The problem
When your SETI@home screensaver downloads a work unit, the data flows from a server in our laboratory, through the University of California at Berkeley campus network, and through a connection to the commercial Internet. This connection is shared by all UCB Internet users - departmental web and FTP sites, email, SETI@home, and so on. The University pays for bandwidth on this connection; it is currently buying 70 megabits per second (Mbps). The student residence hall have a separate 40 Mbps connection.
Until recently, SETI@home was given about 25 Mbps, and the remaining 45 Mbps was shared by the rest of campus. But starting last month (January 2002) the bandwidth used by the rest of campus increased in an unexpected and unexplained way. During peak periods the demand now exceeds 70 Mbps. If SETI@home continued to use 25 Mbps, the performance of all other outgoing traffic would suffer.
The UCB network administrators have worked hard to balance the bandwidth needs of SETI@home and the rest of campus. Currently, SETI@home traffic is given lower priority than other traffic. During peak periods (typically 10 AM - 10 PM PST) SETI@home averages 6 Mbps, and sometimes gets no bandwidth. During non-peak periods SETI@home gets as much as 50 Mbps.
When SETI@home is not getting enough bandwidth, our data server backs up - all of its processes are waiting to send data, and it can't accept new connections. During these periods, your screensaver will get report that it "can't connect to server".
The impact on our overall computing rate is significant but not too serious - the rate has dropped about 25%. But many SETI@home users are unhappy that their computers are sitting idle for many hours, waiting for data. We share this unhappiness, and are working to solve the problem.
Short-term solutions
We're working on several short-term solutions:
Increase the bandwidth of UCB's network connection. We hope to "expand the pipe" by about 10 Mbps - enough to ease, but not eliminate, the crisis. The issue is money - bandwidth costs about $300 a month per megabit, and neither SETI@home nor the university has budgeted for this cost.
Send data more efficiently. Currently work units are encoded as text. By sending them in binary, we can shrink them by about 25%. (Note: data compression isn't effective for our data, which is primarily random noise). This change will require a new version of the client software. Increase the amount of computation per work unit. Doubling the CPU time per work unit - by looking at more chirp rates, for example - will reduce bandwidth by 50%. There is scientific justification for doing this, although the law of diminishing returns applies. This will also require a new version of the client software. Long-term solutions
The long-term solution is to allow work units to be sent from servers outside UC Berkeley. This could be done, for example, by sending work units to servers at organizations - companies and universities - that are willing to donate part of their outgoing network bandwidth to SETI@home. In addition to solving the current problem, this could greatly increase our overall data capacity, enabling us to search for ET signals in a wider frequency band.
This solution represents a significant change to our software; we will use this approach in our next-generation software. We are seeking funding to develop this software, and it won't be ready for at least 6 months.
What you can do There are a couple of things you can do to keep your computers busy processing SETI@home data:
If you connect manually (e.g., over a modem) try connecting during off hours (23:00 to 3:00 Pacific Standard Time, or 7:00 to 11:00 UT). You can check the Server status page to see if we're currently dropping connections. Download more than one work unit when you connect. This can be done manually, or by automated workunit caching software. Example programs include SetiQueue for Windows, or Seti Unit Manager for Macintosh. For more information about other SETI@home add-ons see our links page.
To help us achieve a short-term solution, you can help in two ways:
Donate to SETI@home. This will enable us to buy network bandwidth. Help us find "bandwidth sponsors". We hope that a major commercial ISP might donate bandwidth to UC Berkeley to help SETI@home. If you work for, or have contacts in, such a company, please contact us.
About the current bandwidth problems
If their BW problems stem from the fact that the rest of the campus has experienced a "mysterious" increase in network traffic, a good start may be to block access on ports used by popular file sharing programs. I'll bet that this is where a lot of the BW demand is coming from since the increase happened at the beginning of a new semester.
---
I didn't want to leave this space blank.
You have to give the Set@Home Team their props for making a system thats scaleable and able to handle the user load from the first 100,000 users to the now 3,000,000.
I've always believed the bottleneck in Distributed Computing was the Data Packets being sent/recieved because the demand will grow exponentially the more users you aquire.
Most applications seem to remidy this problem by limiting the data packet sizes from 5 - 15k compressed packets. This has worked for projects like Distributed.net.
I can only forsee the future of this problem being the same that plagues Video Card Chipsets, which is insted of re-engineering the device to make a more robust and lower overhead solution, they'll just throw a bigger pipe on the line (much like Memory Bandwidth demand).
But again, my respect goes out to the Seti@Home team and their sponsors for architecting a technological data mining marvel.
I'm not sure whether or not this is a good thing or a bad thing. Lemme elaborate.
Disclamer: I have never been part of SETI@home; I feel that statistically it's a collossal waste of time. I've been part of both the GIMPS project and the distributed.net RC5-64 projects for about four years now. I've got the Kevlar body armor halfway on.
The good, I guess, is that there's such a collossal interest in this. I mean, hell, if KzAplOcQQ and boB are sharing the Encyclopaedia Galactica (or the Hitchikers' Guide, whatever) over radio waves, then we'll eventually find it hopefully in something that resembles paEr Unicode.
However, I see a great many downsides to this.
First off, if the aforementioned theoretical KzAplocQQ and boB of the paEr race have to use radio waves, then there's a pretty good chance they haven't been able to go superphotonic, in which case we're going to have a long wait before we can even think of going to their New York and flipping them the left tentacle.
Secondly, how will we be able to decode a xenic dataset, much less their language? I mean, what if they can transmit trits or quaytes while we're looking for bits or bytes? How do we know what a newline would appear? Hell, do we even know if it would even be necessary? And what about the characters? What if the Chinese language is easier to interpret than paEr?
Third, there are much better uses of free cycles, at least fiscally. GIMPS will provide a hundred kilobucks to the first person to successfully find a ten megadigit Mersenne prime. distributed.net provides a two kilobuck prize and a large donation to the FSF, EFF, or other worthy charities. Even the commercial distributed computing projects at least pay for the use of your rig.
(PS: paEr is a theoretical name for a xenic (alien) species, contrived from randomly entering characters on the number pad. KzAplocQQ is an unpronouncable name, unless you're lucky or high. boB just sounds funny.)
I used to be someone else. Now I'm someone better.
Real life is underrated.
In a way, this hurdle could prove a boon, by forcing the SETI@home developers to make their system more efficient.
Necessity is, after all, the mother of invention.
As their own statement points out, two of the short-term solutions include making the data sent out more efficient (binary instead of text) and letting each node do more computation.
SETI@home was originally developed to male up for the shortcomings of processing power of any single computer. To solve the problem, they took a bit of a free ride on networking bandwidth to distribute the problem.
Now their success is also forcing them to be more efficient when it comes to network bandwidth, as well as processor, utilization.
So this forced economy will hopefully make the system more efficient through improvement of the system.
Pie-in-the-sky and we have all the computing power and bandwidth we need, but then who would have an incentive to innovate?
Ultimately, SETI@home's legacy will probably have less to do with discoveries of extraterrestrial intelligence and more to do with the evolution of better computing techniques!
evanchik.net
Actually, the network admins have pointed the finger at Kazaa & gnutella. According to the UCB Director of Communications & Network Services, "kazaa and gnutella account for more than half the bits in aggregate". And it's not just SETI that's suffering - all network users have been affected. Unfortunately, a lower priority or outright ban on those services has been rejected due to policy and legal issues.
"she says i'm lousy conversation. as if that's supposed to help."
Go, do it now, I swear you'll feel all warm and fuzzy.
These bandwidth problems aren't technical, they're political. We're getting too close, so they're shutting us down.
* * Always question "the National Interest" - 9 times out of 10 it is a cover for evil
SETI should wait until we have our own world's problems figured out.
Humans are made of meat, and sure, cancer is a problem we'd like to solve. But humans are also uniquely explorers and thinkers, and Not Knowing(tm) IS genuinely one of our problems. Some believe that SETI is a step towards solving that problem. File it under "motivation" or "purpose" (by simplying "knowing").
A future generation may answer the eternal question for us. And if they do, every generation that follows will be affected in their daily outlook, their goals, their attitudes, their comforts, their concerns, etc. That's at least as profound as a cure for cancer.
This wasn't very hard to see coming, but its still unfortunate.
For those who are looking for a workunit-caching program for linux, I've written a perlscript which has done a quite good job at it. I've decided to release it tonight, to help everyone out, but its a bit rough on the edges. It does the job, though. Read the README, download it here. Also, mirrors are welcome - my connection sucks far worse than theirs does =)
Paranoid
Bwaahahahahaa.