Slashdot Mirror

← Back to Stories (view on slashdot.org)

Mailing Disks is Faster than Uploading Data

Posted by CowboyNeal on from the data-access-versus-data-excess dept.

CowboyRobot writes "Who would ever, in this time of the greatest interconnectivity in human history, go back to shipping bytes around via snail mail as a preferred means of data transfer? Jim Gray would do it, that's who. And we're not just talking about Zip disks, no sir. We're talking about shipping entire hard drives, or even complete computer systems, packed full of disks. David Patterson (one of the developers of both RISC and RAID) interviews ACM Turing Award winner Jim Gray." Back in school we always had a saying, "Never underestimate the bandwidth of a station wagon filled with backup tapes." Seems like that still holds true.

25 of 581 comments (clear)

  1. Tapes too... by inertia187 · · Score: 5, Informative

    This reminds me of how data is collected for SETI@Home:

    After the data is recorded onto tapes at Arecibo, they are shipped back to the SETI@home lab in Berkeley, California. The data are then broken up into workunits, which are sent out to the client screensaver program for candidate signal detection. So far, SETI@home has generated 189,598,882 workunits from the data received from Arecibo. SETI@home has split 1,139 tapes, meaning that the average tape yields 166,709 workunits. This is somewhat lower than the optimal yield of roughly 200,000 workunits per tape because of radio frequency interference, gaps in recording, problems with the recording equipment, etc.

    I think a work unit is 65,536 bytes. Even if it takes a week to ship one tape, you can't beat that throughput! But the latency is the worst.

    --
    A programmer is a machine for converting coffee into code.
    1. Re:Tapes too... by pixelite · · Score: 5, Informative
      Well one tape = 166709 units * 64 (k) / 1024 / 1024 = ~10.175GB.


      That figure is per tape, the actual shipment has 1,139 tapes, I think. 10.175GB * 1,139 = ~11.6TB. That *is* impressive bandwith.
      --
      >>Sig under construction
    2. Re:Tapes too... by pclminion · · Score: 5, Insightful
      That figure is per tape, the actual shipment has 1,139 tapes, I think. 10.175GB * 1,139 = ~11.6TB. That *is* impressive bandwith.

      Well in theory, that's not really "bandwidth," it's just a number of bytes. The bandwidth would be the maximum sustained throughput. Essentially, how much data could be delivered per second, if there were a constant stream of trucks pulling in, each carrying 11.6TB. Assume the trucks drive bumper-to-bumper, at 60 MPH. Assume each truck is what, 25' long. At 60 MPH it takes about 0.28 second to travel one truck-length. Therefore, the actual bandwidth is 11.8/0.28 = 42.1 TB per second.

    3. Re:Tapes too... by MyHair · · Score: 5, Funny

      Just try to play Quake3 that way. Actually it starts to help. Play for a while and you start getting kills from rockets you fired 3 maps earlier.

  2. Andy Tannenbaum by Anonymous Coward · · Score: 5, Funny

    "Never underestimate the bandwidth of a station wagon filled with backup tapes."

    "Hurtling station wagon", "8-track tapes".

  3. Wrong Standard! by theGreater · · Score: 5, Funny

    Darn you people! How the heck am I supposed to get a proper astrophysical mental image if you consistently refuse to put things in terms of multiples of VW bugs (the old ones, not the faux ones).

    -theGreater

    1. Re:Wrong Standard! by I+Like+Swords!!! · · Score: 5, Funny

      Sorry, that must be in the metric system or something...

      --
      .unsigged
  4. Netflix by geekee · · Score: 5, Interesting

    Netflix has made a business out of shipping data via snail mail, since the bandwidth isn't really there yet to do it over the internet.

    --
    Vote for Pedro
    1. Re:Netflix by appcoal · · Score: 5, Funny

      Not for very much longer. The RIAA's lawyers have just discovered that the U.S. Postal Service is in fact a p2p network. (Not to mention the highway system.)

  5. We had a saying back in school too... by Capital_Z · · Score: 5, Funny
    We had a saying back in school too --

    "If you're driving a station wagon around you ain't doin' too well with the ladies"

  6. OOoooh high ping by Ribo99 · · Score: 5, Funny

    Of course playing Quake would be out of the question I would think

    --
    I wear pants.
  7. Well, depends on what way you look at it. by gotr00t · · Score: 5, Interesting
    Though it is true that a box filled with hard disks that is snail mailed has a higher rate of transfer than actually uploading the contents of all those hard disks, there are problems with this argument as well.

    First of all, when downloading, you have the benefit of instantly recieving the file that you need, as opposed to waiting at least a day for your shipment to arrive.

    Secondly, remember that bandwidth is probably cheaper than postage. Shipping a carton with a few hard disks and proper insulation would cost at least $30 to overnight it.

    Really, the title of the article comes upon the conclusion way too quickily. You must consider much bandwidth the sender and the reciever have. If both have a several gigabit OC line, then perhaps uploading it would be faster.

    1. Re:Well, depends on what way you look at it. by tarius8105 · · Score: 5, Interesting

      First of all, when downloading, you have the benefit of instantly recieving the file that you need, as opposed to waiting at least a day for your shipment to arrive.

      The average home user still uses a 56k modem, I dont see how it would be faster to transfer a gig on a 56k then priority overnight (much less then 160 gigs).

      Secondly, remember that bandwidth is probably cheaper than postage. Shipping a carton with a few hard disks and proper insulation would cost at least $30 to overnight it.

      Depends on how you ship and what you ship and the connection the person has.

      Really, the title of the article comes upon the conclusion way too quickily. You must consider much bandwidth the sender and the reciever have. If both have a several gigabit OC line, then perhaps uploading it would be faster.

      Not all companies can afford an OC line thus shipping would be cheaper.

    2. Re:Well, depends on what way you look at it. by captain_craptacular · · Score: 5, Insightful

      Lets see, local cable modem is $39.95 for 5 gigs and $10 a gig past that. So if you can ship 3 160GB HDD's for $30 thats:

      160GBX3 = 480GB / $30 = 16 GB/Dollar

      Cable modem = 1GB/$10 = 1/10 GB/Dollar

      So the mail is cheaper. And probably faster if you consider how long it would take to DL 480GB @ 32KB/sec compared to next day or 2nd day air.

      --
      They who would give up an essential liberty for temporary security, deserve neither liberty nor security
  8. Yeah, but the latency's pretty bad... by privaria · · Score: 5, Funny

    PING privaria.org (64.33.49.48) 56(84) bytes of data.
    64 bytes from privaria.org (64.33.49.48): icmp_seq=2 ttl=242 time=2 days, 7 hrs, 37 min
    64 bytes from privaria.org (64.33.49.48): icmp_seq=1 ttl=242 time=2 days, 17 hrs, 14 min
    64 bytes from privaria.org (64.33.49.48): icmp_seq=3 ttl=242 time=3 days, 2 hrs, 41 min

  9. From the interview by grub · · Score: 5, Funny


    Of course, we could put the Library of Congress holdings on it or 10,000 movies

    10,000 movies? The MPAA would like to have a word with him..

    --
    Trolling is a art,
  10. Ping by felonious · · Score: 5, Funny

    The ping on a station wagon sucks and don't even get me started on the routes...

    --
    You aren't free to do anything, until you've lost everything.
  11. I don't know by desenz · · Score: 5, Interesting

    The figures, but does the cost of the bandwidth exceed the price of gas?

    Eh. Guess it doesn't matter anyway. Its still cooler to be seen driving down the street w/ lots of tapes.

  12. Next on /. by Wrexen · · Score: 5, Funny

    This week: You can make a trade-off between latency and throughput!
    Next week: Cars that can haul less can be more fuel-effiecent!
    The week after: Algorithms that use more memory, but are faster to execute!

    Wonders never cease!

  13. The telecom industry is to blame. by jdehnert · · Score: 5, Insightful

    Chips have gotten faster. Ram is bigger faster and less expensive. Disk space is dirt cheap.

    But the telecom industry is just crawling in comparison. I use the same phone line for dial up now as I did 10 years ago, and things like ISDN, DSL, and Cable Modems get you better performance, but nothing stellar. I don't think a T-1 has really changed in cost for a very long time.

    Funny, when the bubble was expanding all the talk was about the bandwidth we were suppored to have access to, but it never made it to my house.

    Eschew Obfuscation

    --
    Eschew Obfuscation
  14. Re:SneakerNet by Anonymous Coward · · Score: 5, Interesting
    I used to work at an IBM site that was used for offsite back-up by major companies. They have these really cool 38 ton trucks that come into the loading bays, where they just connect up a couple of cables and pump the data off the trucks and into the building.

    Basicly they shunt data around, the same way Exxon et al move oil.

  15. The bandwith is there, you just can't have it. by twitter · · Score: 5, Insightful
    Netflix has made a business out of shipping data via snail mail, since the bandwidth isn't really there yet to do it over the internet.

    What a great example you picked! Cable TV companies are pumping dozens of digital movies accross their system at once, live. Yet they crimp your upload speed to DSL rates or lower, 30KB/s, because they are afraid of people "stealing" movies. This is not a technological problem, it a social one. Big publishers and telcos are afraid of competition and are doing everything in their power to keep you from enjoying technology that's already in place. It's the same old fight Ma Bell used to wage back when they would not alow you to so much as plug a modem into your phoneline.

    How long are people here in the US going to put up with this monkey business?

    --

    Friends don't help friends install M$ junk.

    1. Re:The bandwith is there, you just can't have it. by Slurpee · · Score: 5, Informative


      What a great example you picked! Cable TV companies are pumping dozens of digital movies accross their system at once, live. Yet they crimp your upload speed to DSL rates or lower,

      very wrong.

      But enough truth to fool people into believing what you said.

      You are correct in saying that a digital cable system pumps out lots of bandwidth. They do. A movie chan is generally about 4mb/s, possible 8. A chan such as the shopping chan may be 1mb/s. So your cable company with 100 chans is pumping out approx 400mb/s.

      Thats a lot of data.

      But it is broadcast. Each customer is not individually downloading 400mb/s each. They are sharing *one* broadcast. It is not one stream per customer, but one stream is shared between all customers.

      To use a cable for internet, assuming no TV is being broadcast, you can share that 400mb/s between all your users. Customers will have 4kb/s (thats kilobits) EACH (assuming its all shared equally). Not huge.

      Obviously this is not the whole story. Your bandwith is shared between all customers on a node of the cable network (think of them as hubs). If you are the only person in your node, you will get full bandwidth. A node could cover tens, if not hundreds of thousands of users. If every person on your node is using the net to download porn, you will have a very slow connection (better using a modem). Also, the cable company wants to not just do internet, but TV too! In fact, most of the bandwith is used with TV/Movies.

      So, they end up using part of their bandwith for internet, and part for broadcasting TVs.

      How much they set aside for each is a buisness decision, as well as a technology one. If they sell cable internet, the costs are huge, setup, support, network, etc. Costs go up *per user*. Costs for TV is small (ish). Pay for content (movies), get money in from advertising, users, etc etc. No big support costs, no extra costs for bandwith etc etc. One stream can support hundreds of thousands of users.

      It is both a technological problem *and* a buisness problem. They aren't giving you small limits cause they are afraid you will download videos. Don't be paranoid. They don't give you unlimited bandwith cause they can't, and it costs them a lot anyway.

  16. Usenet used to work this way .... by taniwha · · Score: 5, Interesting

    uucp to Australia used to be done by uploading a spool dir somewhere in the US to a tape and airfraighting it to Oz, then doing the same at the other end. You'd post something to usenet and get a reply 2 weeks later

  17. Shipping Laptops by zymurgyboy · · Score: 5, Interesting

    I work in a AmLaw top 100 law firm in DC. We do a lot of complex litigation work. We use software such as Concordance, Ringtail, and Litgator's Notebook (runs on Lotus Notes) to manage collections of documents. The documents are scanned to group IV tiff; the meta data and OCR text that is extracted from the documents at scan time is loaded into another database that overlays the images.

    These tiff file collections run into the millions.

    Of course the point of doing this is to facilitate collaboration on document review between us, our clients and our co-counsel. These people are often 1000s of miles apart, and nearly as often have crap for IT resources (equipment and personnel).

    There are ways of accessing this stuff over the internet securely but it's never quite the same as having the real version of the software. This form of access often proves to be impractical for the lawyers who travel alot depending on the type of access they can get wherever they end up.

    So what often happens is, we end up dumping the entire collection on a laptop with a big hard drive or a bigger firewire or USB drive, so they can work without access to the internet and then replicate changes when they can get the laptop back on ethernet or a POTS line.

    Collections of images and databases (not to mention the various Power Point presentations and word processing files) can very easily run over 50GB. Moving this across the LAN, over my PC BUS to another hard drive and then FEDEXing it is certainly faster than doing the same transfer using FTP or SCP. Not to mention, that way I can install the software (properly) and test the whole setup before I send it off. The extra wear and tear I save on my psyche from NOT having to explain how to install all of the software, point it to the image collections, and deal with equipment I have no control over while being screamed at by extreme Type A attorneys going to trial makes that laptop look like a pretty good investment.

    These are good if you have someone on the other end of your FEDEX run who know how to open the case on a PC and install a HD themselves. I can setup one machine with everything, image the hard drive, make copies on other drives and drop them into FEDEX pouches as fast as I can make 'em. I can't think of a faster way to move a few 100 GBs of stuff to a half dozen places inside of a day. If someone has ideas, I'm all ears.

    --
    If you never make mistakes, it's probably because you're not doing anything.