Slashdot Mirror


Stored Data to Exceed 1.8 Zettabytes by 2011

jcatcw writes "By 2011, there will be 1.8 zettabytes of electronic data stored in 20 quadrillion files, packets or other containers because of, among other things, the massive growth rate of social networks, and digital equipment such as cameras, cell phones and televisions, according to a new study by IDC. Data is growing by a factor of 10 every five years. According to John Gantz, IDC's lead analyst, "at some point in the life of every file, or bit or packet, 85% of that information somewhere goes through a corporate computer, website, network or asset," meaning any given corporation becomes responsible for protecting large amounts of data that it and its customers may not have created. The study, which coincided with the launch of a " digital footprint" calculator, also found that as the world changes over to digital televisions, analog sets and obsolete set-top boxes and DVDs "will be heaped on the waste piles, which will double by 2011.""

32 of 143 comments (clear)

  1. That is a lot of... by sleeping123 · · Score: 5, Funny

    Porn

    1. Re:That is a lot of... by mikael · · Score: 2, Interesting

      Some of the data transfers really seems wasteful. I download a Linux DVD ISO file, burn it onto a DVD, install the system on a new hard disk drive, then download another couple of Gigabytes of updates. Wouldn't be simpler to just have an installation DVD that creates a minimal system which then downloads the latest version of each module.

      And that DVD is really only used once and then forgotten about.

      --
      Vintage computer adverts: http://www.vintageadbrowser.com/computers-and-software-ads
    2. Re:That is a lot of... by beckerist · · Score: 4, Interesting

      From: http://en.wikipedia.org/wiki/Google_platform

      # Upwards of 450,000 servers ranging from a 533 MHz Intel Celeron to a dual 1.4 GHz Intel Pentium III (as of 2005)
      # One or more 80GB hard disks per server (2003)
      So at least using these numbers, let's say on average they have 120gb per server (1 and a half, 80 GB drives...) That would mean they have 54,000 TBs or 54 PBs. I'm sure they have even more now, but as a point of reference! Yes, Google has a finite amount of space!

    3. Re:That is a lot of... by phyrestang · · Score: 5, Informative

      Try installing Gentoo Linux. The current minimal installer for x86 is about 57MB. The rest is downloaded during the installation.

    4. Re:That is a lot of... by Ed+Avis · · Score: 2, Interesting

      I remember when you could do a network install from two floppies...

      --
      -- Ed Avis ed@membled.com
    5. Re:That is a lot of... by NCG_Mike · · Score: 2, Funny

      "Documentaries".

    6. Re:That is a lot of... by stuporglue · · Score: 2, Informative
      You can still do it with one floppy :

      http://damnsmalllinux.org/network-install.html

      • Get TOMSRTBT and boot it
      • Configure network
      • Download install script
      • Download image and use install script
      Debian has a 5-floppy installer still as well : http://ftp.nl.debian.org/debian/dists/etch/main/installer-i386/current/images/floppy/
      --
      https://www.facebook.com/digitizeicm -- Show your support for the digitization of the Iron County Miner newspaper archiv
    7. Re:That is a lot of... by Fission86 · · Score: 2, Funny

      Not to belittle your cause at all, but who still uses floppies any more?

      --
      Coming to you live from another dimension.
    8. Re:That is a lot of... by d3ac0n · · Score: 3, Funny

      Umm.. CD players can be had for as little as $10.00 USD. What's stopping you from getting one?

      --
      Official Heretic from the "Church of Global Warming". Proven right thanks to whistle blowers. AGW = Flat Earth Theory
  2. Riiight by InvisblePinkUnicorn · · Score: 2, Insightful

    "as the world changes over to digital televisions, analog sets and obsolete set-top boxes and DVDs"

    That's what I plan on doing. I'm going to throw out all my DVDs and buy the Blu-Ray equivalent.

    Or maybe I'll just keep the DVDs (and the player) and buy whatever cable adapters I need to get them working on these newfangled devices.

    1. Re:Riiight by Brian+Gordon · · Score: 2, Informative

      What, are you kidding? Blu-ray has horrifying DRM and doesn't really look that much better than DVDs with good postprocessing. I'd never even think of supporting DRMed blu-ray.

    2. Re:Riiight by Tony+Hoyle · · Score: 4, Insightful

      Get a decent TV. There's a massive difference between DVD and Bluray.

      DRM? Who cares. I'm not planning on copying 20gb+ disks.

    3. Re:Riiight by Aenoxi · · Score: 5, Insightful

      Please mod parent up. If I had a nickel for every person who spouted that same upscaled DVD tripe, then, then, then I'd have enough to buy a Blu Ray disk ;)

      There is a world of difference between 1080p and DVD quality - but you'll never see it if your TV can't natively display 1080p (or at least 720) or you use a composite video interconnect rather than HDMI/DVI or component (yes, I know, but you'd be surprised how many people still do...)

      Whilst I can imagine that a true 1080p picture might look similar to upscaled DVD on a small screen (which necessarily has very small dot pitch), the difference becomes clear as you scale up the screen beyond 30 inches or so (and bleeding obvious once you get beyond 42"). Interpolation and post-processing can only get you so far. Notwithstanding CSI, even high-end upscaling cannot create genuine detail that didn't exist in the original image - and the more post-processing you do, the more artifacts you are going to see.

      I've been running a Pioneer BR player via HDMI to a 1080p 60" plasma for 6 months and whilst upscaled DVD is nice, it can't hold a candle to the 1080 BR picture. Double blind test anyone on a similar system and there's no way you'd get anything but a 100% success rate of identifying HD BR vs upscaled DVD.

      --
      "The sum of all knowledge does not imply the knowledge of all sums" Kurt Gödel (paraphrased)
    4. Re:Riiight by meringuoid · · Score: 2, Insightful
      DRM? Who cares. I'm not planning on copying 20gb+ disks.

      I would have said that about DVDs not so long ago. Disk space and bandwidth become cheaper with time.

      And besides copying, a DRM crack allows me to play discs on the operating system of my choice, to extract small parts of the feature for purposes of review, criticism or parody, and to bypass any annoying previews, trailers, propaganda, threats, or other junk that the studio may have seen fit to prepend to the show.

      --
      Real Daleks don't climb stairs - they level the building.
  3. Y2k300! by xZgf6xHx2uhoAj9D · · Score: 5, Funny

    If, like the summary (but not the article for some reason) states, total data is growing by a factor of 10 every 5 years, then somewhere around the year 2300 we'll have 10^80 bits stored. The number of elementary particles in the known universe is estimated to be between 10^79 and 10^81. Seems we're kind of screwed at that point.

    1. Re:Y2k300! by Spad · · Score: 5, Funny

      Just Zip everything, it'll be fine.

  4. Well yes... by theM_xl · · Score: 2, Insightful

    85% of that information somewhere goes through a corporate computer, website, network or asset That's all? I mean, a good deal will be created by corporations in the first place, all the major bits of internet infrastructure belong to one corporation (for-profit or not) or another, the post office is a corporation... 85% seems low, actually.
    1. Re:Well yes... by Dan+East · · Score: 2, Insightful

      I don't know about that. Imagine all of the digital pictures taken that never travel outside the home user's computer, memory card or CDs. Even more important, consider the amount of digital video data generated by home users with their camcorders. A single 60 minute Mini-DV tape is in the neighborhood of 15 GB. That's one single tape, and my family alone has dozens of them just from a single year. Even if those videos are uploaded to the internet, they must first be converted to some other format that has a vastly lower bitrate. So the original gigabytes of data still never touches corporate infrastructure - only the small, crappy quality encodings that end up on YouTube.
      They might also be counting swap files and hibernate files. In the case of hibernate files, a computer with 2 GB RAM generates 2 GB of data every time it hibernates.

      --
      Better known as 318230.
  5. The worse part? by peragrin · · Score: 2, Funny

    Is that half of it will be copies of Windows Vista, XP, a few hundred Linux distro's.

    --
    i thought once I was found, but it was only a dream.
  6. Which definition of a zetabyte? by EricR86 · · Score: 4, Interesting

    Since we're talking very large orders of magnitude it would help to know what definition of zetabyte they're using.

    2^50 bytes or 10^15 bytes?

    The former is astronomically larger.

    1. Re:Which definition of a zetabyte? by EricR86 · · Score: 2, Informative

      2^50 bytes or 10^15 bytes? What I really meant was: 2^70 bytes or 10^21 bytes? Pfft. Only a few orders of magnitude... :|
    2. Re:Which definition of a zetabyte? by pipatron · · Score: 4, Funny

      If by "astronomically larger" you mean 12.6%, then I'm astronomically larger than the average Indonesian male.

      --
      c++; /* this makes c bigger but returns the old value */
    3. Re:Which definition of a zetabyte? by sapphire+wyvern · · Score: 2, Insightful

      At the risk of being modded down, isn't that distinction the whole point of the IEC's "zebibyte" proposal?

      Anyway, most measurements of mass storage (bandwidth quotas, hard disk capacity etc) seem to measured in actual megabytes (MB), gigabytes (GB) etc, as opposed to binary megabytes (MiB), binary gigabytes (GiB) and so on. Binary byte prefixes only seem to be used for RAM and flash these days, presumably because of the convenient manufacturing realities involved - and I really wish that manufacturers of those products would get with the program and label their products with unambiguous units.

      So I assume the estimate means 10^15 bytes.

    4. Re:Which definition of a zetabyte? by TheRaven64 · · Score: 2, Insightful

      In theory, yes. In practice, the whole Zebibyte thing is complete nonsense. Everyone other than hard drive manufacturers has been using the SI prefixes to refer to power of two quantities when referring to binary data for 40 years. Attempting to redefine them retroactively just causes confusion. If I see something that says KB, and don't know when it was written, I have no idea if it pre or post-dates the KiB nonsense and so I have no idea if it refers to 1024 or 1000 bytes.

      --
      I am TheRaven on Soylent News
    5. Re:Which definition of a zetabyte? by xaxa · · Score: 2, Insightful

      So you're better off if someone does use the proper prefix then. Without it, KB could mean either. With it, at least you know what kiB means, so you're definitely right some of the time.

    6. Re:Which definition of a zetabyte? by Waffle+Iron · · Score: 3, Insightful

      Everyone other than hard drive manufacturers has been using the SI prefixes to refer to power of two quantities when referring to binary data for 40 years. Attempting to redefine them retroactively just causes confusion.

      No, the confusion is cause by using a pseudo-binary based number system in a world where almost everything else is decimal.

      Quick question: You have a 2000 MiB video file and a 2470 MiB video file. Will they both fit on a 4.37 GiB DVD? Now you need your calculator.

      It's much easier to figure out if a 2097 MB and a 2590 MB file fit on a 4.7 GB disk. You can do that in your head.

      I've been burned numerous times by programs ambiguously reporting sizes in KiB and MiB causing me to run out of space on something that I'm trying to fill. All storage sizes should always be reported in decimal numbers. If RAM manufacturers want to keep using powers of two due to the implementation detail of how their chips are constructed, they should *always* use KiB, MiB and GiB.

    7. Re:Which definition of a zetabyte? by Waffle+Iron · · Score: 2, Insightful

      For everything else, that is, using a computer, it's back to binary.

      It is not. RAM is the only quantity in computers commonly measured in binary. Hard drives have always been in decimal. Floppies have always been in an even more stupid system where "MB" == 1000*1024. Clock speeds have always been decimal.

      Going farther, measuring IO or network performance, to cite two trivial examples, or understanding any of those subjects in general, you're binary to binary.

      You appear to have been bambooozled yourself by the confusion caused by this issue. I/O speed of buses is always decimal because it derives from MHz and GHz, which are decimal. Network bandwidth is more often measured in decimal megabits, not binary.

      You seem to think that just because one user app, Windows Explorer, confusingly shows binary based quantities, then everything else in the computer is or ought to be measured that way as well. You're incorrect.

      I don't see why learning powers of two, and then extending that (for the "power users") to base 16, is unreasonable.

      If you were advocating that people learn and work in pure hexadecimal, you might have a point. However, these units aren't a consistent radix. They're a strange mishmash of binary and decimal based on the accident that 2**10 is somewhere close to 10**3. They have completely different math for each of KiB, MiB, GiB, etc. You're telling people that they need to work with four or more distinct new number systems, and be prepared to convert between any and all of them, depending on approximately how much data they're working with. That's just stupid.

  7. Re:But what we really want to know is.... by oni · · Score: 4, Funny

    no no no, the proper term for journalists to use is library of congresses. Even though I've never been to the library of congress and have no idea how big or small it might be, large amounts of data should always be given in those units.

  8. Wrong metric? by guruevi · · Score: 3, Interesting

    I was wondering if they weren't a bit wrong in their calculations. A Zettabyte is 1 Million Petabytes. Knowing that where I work has about 2 petabytes in a few SAN's and there are 1000's of larger institutions and millions that are smaller (that store in the terabytes range) around the world. The place I worked before had about a half a petabyte just in tape backups for credit card and other transactions, catalog and pricing information, images etc. and that was just an average clothing company, hardly rivaling JCPenney or Macy's. I'm also thinking about Wal-Mart with millions of products and thousands of stores. And we're just talking about SAN's here mainly in the US, not including desktops, laptops, camera's, personal information, Google.

    On another note, how much does a zettabyte actually yield these days, drive manufacturers might just give you 700 Petabytes for it. Oblig. XKCD: http://xkcd.org/394/

    --
    Custom electronics and digital signage for your business: www.evcircuits.com
  9. Data figures are misleading by Bombula · · Score: 3, Insightful
    The interesting thing here is the part about data being relayed through third parties and the issues involved. As for the data figures themselves, those are pretty misleading because data does not equal useful information. There is far less useful information in an MS Word file than 100Kb or whatever, for example, so these zetabyte figures bandied about aren't terribly meaningful other than to draw attention to the infrastructure needed to support digital data relaying. To see my point, turn things upside down: there is vastly more data stored on an LP record or celluloid film than on a CD or digital photograph. But is that data useful information? Only a few audiophiles and filmophiles would argue that there is.

    Yes, there is a lot of data in the world. But is there really that much more information out there? A zillion copies of the same song just means more data, not more information.

    --
    A-Bomb
  10. Re:You are answering yourself by epine · · Score: 3, Interesting

    secondly, who really cares? Most of it is cached google pages and pron anyway... That's why /.ers care. But actually, no. We're very close already to being able to generate pron on demand without involving any principle photography. You won't even need to say what you want, that will be ascertained on the fly by neuro-cranial-bio-feedback.

    After enough of the male population has been brain mapped, it will probably turn out like spam: there's only so many unique permutations, as long as the scene is dressed up a little differently from time to time to maintain the novelty factor.

    Pron seems to be a lot like Big Bertha, where each mortar round was larger than the last, to accommodate progressive barrel enlargement. Eventually the images become extremely shocking to get any response at all.

    http://www.wired.com/science/discoveries/news/2008/03/mri_vision

    The future of compression is not to send the picture itself, but the reduced specification for an image that produces the same effect on the human visual system. We're already doing this with psycho-acoustic encoding.

    Once we have a sufficiently sophisticated model of human sensory perception, mental and emotional responses (which will run to TBs I'm sure), we can run a competition for the best feature movie encoded in under 4KB. Mostly it would describe desired emotional responses and cognitive states, the actual images would be back-generated to achieve this effect as determined by the human perceptual model.
  11. 4 Gigs. by Warll · · Score: 2, Insightful

    So I used their digital foot print calc, it told me mine was 4.5 gigabytes. A little on the low side I'd say, I have 1.1 TB of HDD sitting right next to me.