Slashdot Mirror


'Digital Universe' To Add 1.8 Zettabyte In 2011

1sockchuck writes "More than 1.8 zettabytes of information will be created and stored in 2011, according to the fifth IDC Digital Universe study. A key challenge is managing this data deluge (typified by the Large Hadron Collider at CERN, which generates 1 petabyte of data per second)."

44 of 60 comments (clear)

  1. LHC data is _not_ stored in the digital universe by rbrausse · · Score: 3, Insightful

    the experiments may generate the PB per second but most of the data is rejected before it hits any storage system...

  2. 1.8 Petabyte per second... by LordofEntropy · · Score: 1

    Wow that's a lot of data. Can't wait to see more of the results published.

    --
    Entropy just isn't what it used to be.
    1. Re:1.8 Petabyte per second... by cashman73 · · Score: 1

      How many libraries of congress is that?

    2. Re:1.8 Petabyte per second... by daem0n1x · · Score: 1

      And 99,9999% of it is completely useless.

    3. Re:1.8 Petabyte per second... by perryizgr8 · · Score: 1

      its a decimal, not a comma. why do so many people make the same stupid mistake?

      --
      Wealth is the gift that keeps on giving.
    4. Re:1.8 Petabyte per second... by daem0n1x · · Score: 1

      Stupid is to think English is the only language in the world.

    5. Re:1.8 Petabyte per second... by perryizgr8 · · Score: 1

      decimal notation changes with language?!? that's a new one. even if it does, the rest of your comment WAS in english.

      --
      Wealth is the gift that keeps on giving.
    6. Re:1.8 Petabyte per second... by daem0n1x · · Score: 1

      decimal notation changes with language?!?

      Yes it does.

      that's a new one.

      Only for you

      even if it does, the rest of your comment WAS in english.

      The decimal separator on my keypad is comma. If you weren't so swift in calling me stupid, I would have explained it earlier.

    7. Re:1.8 Petabyte per second... by perryizgr8 · · Score: 1

      ok, my bad.

      --
      Wealth is the gift that keeps on giving.
  3. Hellagood by slipnslidemaster · · Score: 1

    That's 1.8 x 10^-6 hellabytes for those of you keeping track.

    --


    "What the hell is an aluminum falcon?"
    1. Re:Hellagood by BrokenHalo · · Score: 1

      That unit is as good as any. We're rapidly running out of prefixes here, and we still need formal definitions for units such as bucketload, shitload, shitton[ne] and so forth.

    2. Re:Hellagood by slipnslidemaster · · Score: 1

      The key to getting a prefix established...is to just start using it.  The rest of the world and ultimately standardizing bodies will adopt it eventually.

      http://www.facebook.com/pages/The-Official-Petition-to-Establish-Hella-as-the-SI-Prefix-for-1027/277479937276?ref=search&sid=1050298974.3729275378..1

      --


      "What the hell is an aluminum falcon?"
    3. Re:Hellagood by BrokenHalo · · Score: 1

      Fair enough, but I doubt if you're going to win any converts by insisting on posting your comments with horrible monospaced fonts just to grab attention... :-}

  4. Re:LHC data is _not_ stored in the digital univers by Infiniti2000 · · Score: 3, Insightful

    Indeed, only about 25PB are stored every year from the LHC.

  5. Re:LHC data is _not_ stored in the digital univers by Joce640k · · Score: 1

    Shouldn't it be "Pebibytes"? We're supposed to be geeks.

    --
    No sig today...
  6. Can I have that in LoCs by Bob+the+Super+Hamste · · Score: 4, Funny

    Can we get that in a proper measurement like Libraries of Congress.

    --
    Time to offend someone
    1. Re:Can I have that in LoCs by Phurge · · Score: 1

      Using 1 LoC = 20 TB, then 1.8 ZB = 96,636,764 LoCs

      Or as wolfram alpha says 1.8 ZB = 144,000 x estimated information content of all human knowledge.

      --
      I'll see your hokum and raise you a boondoggle.
    2. Re:Can I have that in LoCs by Phurge · · Score: 2

      oops 1.8 ZB = 144 x estimated information content of all human knowledge.

      --
      I'll see your hokum and raise you a boondoggle.
    3. Re:Can I have that in LoCs by Cylix · · Score: 1

      Or as woflram and heart says, 0 = 144,000 x 0 (estimated worth of human knowledge).

      --
      "You should always go to other people's funerals; otherwise, they won't come to yours." -- Yogi Berra
  7. Re:LHC data is _not_ stored in the digital univers by Daimanta · · Score: 1

    Shouldn't it be "Pebibytes"? We're supposed to be geeks.

    Yes, but we're not morons.

    --
    Knowledge is power. Knowledge shared is power lost.
  8. How much is redundant by Bob+the+Super+Hamste · · Score: 2

    I wonder how much of that data is redundant. I know that for one of my side projects I have "redundant" data that I got from the Minnesota DNR, various MN counties, the state legislature, and the federal gov. Even after it had been preprocessed and trimmed down so it only has what I care about it is still around 12GB of vector data which is about 1/3 the original size.

    --
    Time to offend someone
  9. bah by PJ6 · · Score: 1

    that's just Netflix.

  10. 57.5 billion 32GB iPads required to store this :D by Mightee · · Score: 1

    to store this amount of data, you need 57.5 billion 32GB iPads which will cost around $34.4 trillion — and that's equivalent to Gross Domestic Product (GDP) of United States, Japan, China, Germany, France, the United Kingdom and Italy combined. :D

  11. Re:Free Dating Websites by L4t3r4lu5 · · Score: 1

    rel=nofollow dude. Your linkwhoring is pointless here.

    --
    Finally had enough. Come see us over at https://soylentnews.org/
  12. Great by Dunbal · · Score: 1

    More space to be filled by Russian mining bots. Oh wait which universe is this?

    --
    Seven puppies were harmed during the making of this post.
  13. Re:large size by L4t3r4lu5 · · Score: 2

    Don't worry, the large size won't be an issue. You can put it in a ZIP and then put that into another ZIP and so on.

    That's just stupid. They use the same compression algorithm!

    Put it in a ZIP in a TAR in a RAR in a 7z in an ACE in a bZip in a CAB in a dmg in a a ARJ, and finally save it as a GIF. You can't use JPEG as it's lossy.

    --
    Finally had enough. Come see us over at https://soylentnews.org/
  14. Large Hadron Collider data anomaly? by L4t3r4lu5 · · Score: 3, Interesting

    So it generates 1PB of data per second, yet from the article "[T]he data comes from the four machines on the LHC in which the collisions are monitored â" Alice, Atlas, CMS and LHCb â" which send back 320MB, 100MB, 220MB and 500MB"

    That's a few orders of magnitude short of 1 Petabyte, folks. Where are these numbers coming from?

    --
    Finally had enough. Come see us over at https://soylentnews.org/
    1. Re:Large Hadron Collider data anomaly? by Eivind · · Score: 1

      One is the raw-amount, simply the number of sensors multiplied with the frequency each sample at multiplied with the size of each sample.

      But the article itself say they filter and store only the interesting stuff, which is, as we can see from the later numbers, a triflingly small fraction of the entirety.

      A camera that can do full-HD at 30fps captures 186MB/s afterall, but it does not follow that a facility with 3 such security-cameras need to store however many petabytes that becomes, in order to have a recording of the burglars.

    2. Re:Large Hadron Collider data anomaly? by L4t3r4lu5 · · Score: 1

      Those numbers add up to the 25PB/year that they store. The other 99.9999% (really) is filtered our at the detectors before being "sent back".

      Ah, thanks for that. I was under the impression that the computer centre would be discarding the unused data, but if the detectors are smart enough to do it then all the better!

      --
      Finally had enough. Come see us over at https://soylentnews.org/
    3. Re:Large Hadron Collider data anomaly? by AJH16 · · Score: 1

      As eivind pointed out, the PB a second is raw data. The LHC utilizes 2 layers of in hardware filtering and another layer of software filtering (as I recall from a while back at least) in order to trim the data down to a quasi-reasonable data stream that can be effectively stored and analysed.

      --
      AJ Henderson
    4. Re:Large Hadron Collider data anomaly? by MartinSchou · · Score: 2

      If you drive 60 miles/hour for 30 seconds, you haven't driven 60 miles. One is a measure of speed, the other is distance.

      Same with this. 1 PB/s is speed. 1,140 MB is the amount of data. All it really means is that these 1,140 MB are generated (and possibly collected) in 1.06 microseconds.

  15. Re:LHC data is _not_ stored in the digital univers by AliasMarlowe · · Score: 1

    Indeed, only about 25PB are stored every year from the LHC.

    No. They store all of it, but mostly in /dev/null

    --
    Those who can make you believe absurdities can make you commit atrocities. - Voltaire
  16. Wow, great compression at the LHC! by brian0918 · · Score: 1

    (typified by the Large Hadron Collider at CERN, which generates 1 petabyte of data per second)

    So, the LHC produces 1 petabyte per second, and given that there are 30+ million seconds in the year, that means the LHC produces 30+ zettabytes a year. Clearly there is a problem with your typification.

  17. Bogus by vlm · · Score: 1

    I call bogus on this.

    10e21 / 10e10 = 10e11 bytes/living human being.

    The global GDP and global hard drive manufacturers simply cannot support a 100 GB hard drive per person per year... Cheapest option per byte is probably 1 TB drive for every 10 people. My basement therefore balances against a small African village, but there's plenty of small African villages, and only one me.

    Even if all the ACTIVE /.-er types have a basement like mine, and they do not, there are simply not enough of us. And on a global GDP basis a tenth of a TB hard drive is way too expensive per person, that would put data storage at roughly "rice consumption" levels. And the rich are only getting richer while the poor get poorer for some decades now, so don't try the "world is getting richer thus can afford it" argument.

    --
    "Science flies us to the moon. Religion flies us into buildings." - Victor Stenger
    1. Re:Bogus by stevelinton · · Score: 1

      Do the sums. 1 ZB = 10^9 TB. a TB had drive costs c US$50, probably less in quantity, so information storage is US$50 billion/year industry.
      Doesn't seem implausible to be honest.

    2. Re:Bogus by weffew... · · Score: 1

      Most of CERN's data isn't on hard drives anyway - it's got the biggest tape system in Europe that I know of (I do high end storage for a living).

      A photo, for funsies: http://www.flickr.com/photos/naezmi/3309812634/

      C

  18. Re:large size by Noughmad · · Score: 1

    No, man. It's just ZIP's all the way down.

    --
    PlusFive Slashdot reader for Android. Can post comments.
  19. Re:Well that's inconvenient... by tom17 · · Score: 1

    Why two? For redundancy?

  20. Re:LHC data is _not_ stored in the digital univers by AliasMarlowe · · Score: 1

    And as geeks we understand that the English language isn't governed by a committee of Swiss engineers.

    Unfortunately, they're English engineers...

    --
    Those who can make you believe absurdities can make you commit atrocities. - Voltaire
  21. Re:57.5 billion 32GB iPads required to store this by networkBoy · · Score: 1

    You are right, of course.
    That'll be 1,500,000,000,000 double density 8 inch floppies worth of data. Much more efficient, and at the last price I paid for them, (about 12 years ago now) they were $12.00 each (not per box, each)) so that is $18,000,000,000,000, or roughly 52.325581395348837209302325581395% of the cost of the iPads.
    -nB

    --
    whois gawk date unzip strip find touch finger mount join nice man top fsck grep eject more yes exit umount sleep dump
  22. LHC data will not be information for years to come by recharged95 · · Score: 1

    And most of it will be junk data.

    Data != Information. And then there's Metcalfe's law applied to information value.

  23. Re:LHC data is _not_ stored in the digital univers by the_other_chewey · · Score: 1

    Yes, and as geeks we understand that the quoted number is an approximate
    one and the data generated are not inherenly binary, so there is no need
    for either the precision of "exactly one PiB" or the context of "this is binary".

    In fact, the decimal prefix is the much more sensible one to use here.

  24. Re:LHC data is _not_ stored in the digital univers by GrumpySteen · · Score: 1

    I suppose we should be grateful that data isn't measured in petahogsheads.

  25. Re:Question about slashdot by perryizgr8 · · Score: 1

    and why in hell does a /. tab not close immediately but does some shit for a few miiliseconds before closing?

    --
    Wealth is the gift that keeps on giving.