Slashdot Mirror

← Back to Stories (view on slashdot.org)

The 1-Petabyte Barrier Is Crumbling

Posted by CmdrTaco on from the so-much-data dept.

CurtMonash writes "I had been a database industry analyst for a decade before I found 1-gigabyte databases to write about. Now it is 15 years later, and the 1-petabyte barrier is crumbling. Specifically, we are about to see data warehouses — running on commercial database management systems — that contain over 1 petabyte of actual user data. For example, Greenplum is slated to have two of them within 60 days. Given how close it was a year ago, Teradata may have crossed the 1-petabyte mark by now too. And by the way, Yahoo already has a petabyte+ database running on a home-grown system. Meanwhile, the 100-terabyte mark is almost old hat. Besides the vendors already mentioned above, others with 100+ terabyte databases deployed include Netezza, DATAllegro, Dataupia, and even SAS."

6 of 217 comments (clear)

  1. Google Maps is way bigger... by Plantain · · Score: 3, Informative

    Google Maps' database is far bigger...

    A base of 8 tiles, with each becoming four more smaller tiles, in two modes (map/satellite), and 16 zoom levels.

    Each tile is approx. 30kB.

    (((0.03* (8 * (4^16)))/1024)/1024) == 983.04TB right there.

    My calculator doesn't handle numbers big enough for streetview. O_O

    --
    No, but I did throw granola at a deaf person once
  2. Science! by edremy · · Score: 5, Informative
    Petabytes are actually pretty common in the sciences. I visited NCAR (National Center for Atmospheric Research) in Boulder five years ago and their main database was in the 2PB region even then. I'm sure it's a lot larger today

    The LHC will generate several PB of data per year, as will the Large Synoptic Survey Telescope. These projects aren't all that uncommon.

    --
    "Seven Deadly Sins? I thought it was a to-do list!"
  3. Re:I am confused !! by Anonymous Coward · · Score: 4, Informative

    1 Petabyte = 1,000 Terabytes
    1 LoC = 10 Terabytes
    100 LoC = 1,000 Terabytes
    ======
    100 LoC = 1 Petabyte

  4. Re:Google Street View must be most massive db ever by Anonymous Coward · · Score: 5, Informative

    http://labs.google.com/papers/bigtable.html

  5. WalMart has a 4 petabyte database already by captaindomon · · Score: 4, Informative

    WalMart's data warehouse is already 4 petabytes: http://storefrontbacktalk.com/story/080307walmart.php

    --
    Just because I can hook a shark from a boat, I do no offer to wrestle it in the water.
  6. LHC data production by SlowMovingTarget · · Score: 3, Informative

    So when active, the Large Hadron Collider will generate the equivalent volume of data of 50 Libraries of Congress every second.