Slashdot Mirror
Mailing Disks is Faster than Uploading Data
CowboyRobot writes "Who would ever, in this time of the greatest interconnectivity in human history, go back to shipping bytes around via snail mail as a preferred means of data transfer? Jim Gray would do it, that's who. And we're not just talking about Zip disks, no sir. We're talking about shipping entire hard drives, or even complete computer systems, packed full of disks.
David Patterson (one of the developers of both RISC and RAID) interviews ACM Turing Award winner Jim Gray." Back in school we always had a saying, "Never underestimate the bandwidth of a station wagon filled with backup tapes." Seems like that still holds true.
This reminds me of how data is collected for SETI@Home:
After the data is recorded onto tapes at Arecibo, they are shipped back to the SETI@home lab in Berkeley, California. The data are then broken up into workunits, which are sent out to the client screensaver program for candidate signal detection. So far, SETI@home has generated 189,598,882 workunits from the data received from Arecibo. SETI@home has split 1,139 tapes, meaning that the average tape yields 166,709 workunits. This is somewhat lower than the optimal yield of roughly 200,000 workunits per tape because of radio frequency interference, gaps in recording, problems with the recording equipment, etc.
I think a work unit is 65,536 bytes. Even if it takes a week to ship one tape, you can't beat that throughput! But the latency is the worst.
A programmer is a machine for converting coffee into code.
i believe your attribution is correct.
Never underestimate the bandwidth of a station wagon filled with backup tapes
however, while the immediate bandwidth of a station wagon filled with tapes may be enormous, the overall bandwidth is quite poor. this is because of the slow write/read rates of the tape drive, and the slow overall speed of the station wagon. i can transfer 3 gigs from my work computer to my home machine faster than the time it would take me to write the 3 gigs to tape, drive it there, and read it back from tape (and my drive is only 15 minutes). if i lived 5000 miles away, my tape bandwidth would be considerably worse, while my internet bandwidth would be virtually unchanged.
since this statement was made, we have reached the point where internet bandwidth has exceeded the "vehicle full of tapes". now, this one might be good for a few more years:
Never underestimate the bandwidth of an sr71 full of netapps
The station wagon comment reminded me of an idea that I had a long time ago, when I first read about how the Internet routes packets around. You know how you can ship stuff UPS overnight? It can get pretty expensive, depending on how big and heavy the package is. And sometimes, businesses would pay an even greater price to have a package delivered even faster. Why not introduce a system for getting things delivered extremely fast, and I do mean fast, all around the world?
Imagine this: Put together a network of railroad-like tracks that are enclosed in concrete tunnels. In a vacuum. Individual cars would travel on these tracks at greater than mach speeds. They would essentially go from one switching station to another, kind of like the telephone network or the Internet. They might come in several sizes, these cars. When you need something delivered fast from, say Los Angeles to New York, the package would be placed on a dedicated car which would take it at blazing speeds through, say, Albuquerque, Oklahoma City and Louisville, to New York. At each station, equipment would adjust switch tracks to route the car to its next switching station; the car would not even have to stop or slow down. The package might be there in four hours, counting the time it takes to bring the package to a station, have it loaded, unloaded, and then transporting it to its final destination.
This might actually make shipping cheaper rather than more expensive. Automatic equipment sorts mail at the USPS. If this mail were collected, say, once every hour (during business hours), taken to the nearest major USPS distribution center, where it is sorted, placed in boxes heading to the same destinations, and then shipped (tunneled?) through the above method, mail going to a distant location might arrive faster than mail going across town. This could be done with collections of packages that are all going from one major city to another together. Load them in a container and bust them all over there. Sure, it'll still take, say, 24 hours to ship packaged in such groups, to save money, since you have to wait for enough packages, sort them, group them, etc., but if you want something shipped right friggin now, the option to get a dedicated car is still available. This might reduce use of gasoline and use of air and ground traffic. If computers can control the cars on these tracks so that cars are going mach 2 almost bumper to bumper, that would allow for extremely great throughput.
Back to the station wagon comment, supposing this could be done, (running more tracks all over the world and installing these switching stations at each major city), you could load hundreds of terabytes of data onto a big friggin raid system and then get that data across the world faster than shit going through a tin horn.
What a great example you picked! Cable TV companies are pumping dozens of digital movies accross their system at once, live. Yet they crimp your upload speed to DSL rates or lower,
very wrong.
But enough truth to fool people into believing what you said.
You are correct in saying that a digital cable system pumps out lots of bandwidth. They do. A movie chan is generally about 4mb/s, possible 8. A chan such as the shopping chan may be 1mb/s. So your cable company with 100 chans is pumping out approx 400mb/s.
Thats a lot of data.
But it is broadcast. Each customer is not individually downloading 400mb/s each. They are sharing *one* broadcast. It is not one stream per customer, but one stream is shared between all customers.
To use a cable for internet, assuming no TV is being broadcast, you can share that 400mb/s between all your users. Customers will have 4kb/s (thats kilobits) EACH (assuming its all shared equally). Not huge.
Obviously this is not the whole story. Your bandwith is shared between all customers on a node of the cable network (think of them as hubs). If you are the only person in your node, you will get full bandwidth. A node could cover tens, if not hundreds of thousands of users. If every person on your node is using the net to download porn, you will have a very slow connection (better using a modem). Also, the cable company wants to not just do internet, but TV too! In fact, most of the bandwith is used with TV/Movies.
So, they end up using part of their bandwith for internet, and part for broadcasting TVs.
How much they set aside for each is a buisness decision, as well as a technology one. If they sell cable internet, the costs are huge, setup, support, network, etc. Costs go up *per user*. Costs for TV is small (ish). Pay for content (movies), get money in from advertising, users, etc etc. No big support costs, no extra costs for bandwith etc etc. One stream can support hundreds of thousands of users.
It is both a technological problem *and* a buisness problem. They aren't giving you small limits cause they are afraid you will download videos. Don't be paranoid. They don't give you unlimited bandwith cause they can't, and it costs them a lot anyway.