Slashdot Mirror

← Back to Stories (view on slashdot.org)

HPE Unveils The Machine, a Single-Memory Computer Capable of Addressing 160 Terabytes (venturebeat.com)

Posted by BeauHD on from the era-of-big-data dept.

An anonymous reader quotes a report from VentureBeat: Hewlett Packard Enterprise announced what it is calling a big breakthrough -- creating a prototype of a computer with a single bank of memory that can process enormous amounts of information. The computer, known as The Machine, is a custom-built device made for the era of big data. HPE said it has created the world's largest single-memory computer. The R&D program is the largest in the history of HPE, the former enterprise division of HP that split apart from the consumer-focused division. If the project works, it could be transformative for society. But it is no small effort, as it could require a whole new kind of software. The prototype unveiled today contains 160 terabytes (TB) of memory, capable of simultaneously working with the data held in every book in the Library of Congress five times over -- or approximately 160 million books. It has never been possible to hold and manipulate whole data sets of this size in a single-memory system, and this is just a glimpse of the immense potential of Memory-Driven Computing, HPE said. Based on the current prototype, HPE expects the architecture could easily scale to an exabyte-scale single-memory system and, beyond that, to a nearly limitless pool of memory -- 4,096 yottabytes. For context, that is 250,000 times the entire digital universe today.

24 of 150 comments (clear)

  1. "The Machine" could they get any more non-descript by UnknownSoldier · · Score: 5, Interesting

    > it could require a whole new kind of software.

    Huh? You mean it not a von Neumann or Harvard architecture because the article doesn't lead me to _that_ conclusion:

    The new prototype has 160 TB of shared memory spread across 40 physical nodes, interconnected using a high-performance fabric protocol. It has an optimized Linux-based operating system (OS) running on ThunderX2, Caviumâ(TM)s flagship second generation dual socket capable ARMv8-A workload optimized System on a Chip.

    So basically 4 TB / node. Is each node have independent memory or not?
     

  2. Just great. by fahrbot-bot · · Score: 5, Funny

    I'll have to allocate an entire 1.6 TB drive for swap space.

    --
    It must have been something you assimilated. . . .
  3. Re:"The Machine" could they get any more non-descr by imgod2u · · Score: 2

    I would wager to guess that each node lives in some subregion of the memory address. And that each OS instance (or one giant distributed OS) accesses all addresses uniformly.

    It's certainly not infeasible even without memristor tech. But I wonder what benefits it has. The whole point of having localized nodes is to take advantage of the travel latency. Unless this is optimized specifically for embarrassingly parallel data feed-forward tasks, which even modern GPU workloads aren't anymore.

  4. Re:Does is Run Linux? by rudy_wayne · · Score: 5, Funny

    Finally, enough RAM for Firefox!!

  5. I might regret saying this but... by gfilion · · Score: 5, Funny

    160 TB of RAM ought to be enough for anybody

  6. Re:"The Machine" could they get any more non-descr by imgod2u · · Score: 4, Interesting

    It seems to imply more than just persistent memory, though. It sounds like they're distributing processors in the data-path of the connected memory. Instead of the OS determining which context to put on a CPU and fetching the necessary data from memory/disk, the context and code will be decided by what data resides in memory that is closest to the processor node.

    A rather natural result of persistent, high-capacity memory for non-interactive compute tasks.

  7. Ob by Hognoxious · · Score: 4, Funny

    It's almost enough to store all the data their keylogger stole.

    --
    Confucius say, "Find worm in apple - bad. Find half a worm - worse."
  8. Bus size by Steve-Oh · · Score: 2

    4096 yottabytes = 4.096e27 bytes; 2^n=4.096e27, solve for n ... n = 92. Now we know the market for these 128-bit processors!

  9. Interesting but, not amazing by somenickname · · Score: 5, Insightful

    It would have been a lot more interesting, and a lot more paradigm shifting, if it was 160TB of ultra-fast next-gen M.2 sticks with 0MB of traditional RAM and 0MB of traditional storage. That would be a truly unique machine to work on. If you read the article, this isn't even a single machine. It's actually 40 nodes with high speed interconnects. Basically, HP is now running Linux on their VMS clusters.

  10. Re:Does is Run Linux? by Anonymous Coward · · Score: 5, Funny

    But it still comes short of what Chrome needs.

  11. Re:"The Machine" could they get any more non-descr by dbIII · · Score: 2

    But I wonder what benefits it has

    Being able to do an operation on an entire huge dataset in memory instead of a pile of fetching and carrying to do it on disk.
    Since the alternative is an order of magnitude (or several) slower a bit of latency isn't a terrible price to pay.

  12. Re:"The Machine" could they get any more non-descr by knightghost · · Score: 2

    AI using multidimensional data sets. I work with cubes in the tens of terabytes that could be sped up thousands of times if they could be held in memory.

  13. Re:Remember...this is HP by MancunianMaskMan · · Score: 3, Insightful
    years ago we heard HPE (or was it still HP then) talked about betting the farm on "the machine" all full of its new memristor tech, cheap, fast, persistent, practical, egg-laying wolly milk pig kind of chips.

    Now it's "DIMMs with a little battery stuck on" to handle the "persistency". Hope that's just for the demo.

  14. Re:Does is Run Linux? by KiloByte · · Score: 2

    Does it run Linux? That's the first question.

    Only.

    The second, is this like 10 years out?

    Multiple vendors sell servers with 64TB RAM already, and expanding further was blocked by the lack of 5-level paging. Patches to do so have been floating on LKML for a while, thus hardware that can do that should be well past prototype stage.

    On the other hand, all patches I've seen are for x86, and this is arm64, so I'm apparently missing something.

    --
    The creatures outside looked from Alt-Right to Antifa; but already it was impossible to say which was which.
  15. Re:"The Machine" could they get any more non-descr by Maritz · · Score: 2

    Basically they took 10 PCs and put the PC boxes in another box, then labelled that box "The Machine". A box of boxes. It'll change the world!

    That's your take-home from this? lol.

    Stick to playing with the worms in your garden mate.

    --
    I do not want your cheap brainburning drugs. They are useless for work. And I am a working man today.
  16. Books? by mark_reh · · Score: 2

    Seriously, are we still using books as a unit of comparison? Why not say it can process 80% of the internet, etc.?

    1. Re:Books? by Voyager529 · · Score: 3, Informative

      Seriously, are we still using books as a unit of comparison? Why not say it can process 80% of the internet, etc.?

      Yes, and there are two related reasons. First, the LoC is a very large amount of data. It's not the kind of data that can land on a USB stick, it's enough to actually prove something.

      Second, it's a known quantity of data. Even if it's approximate, it's a set amount of books, with a set amount of pages. Can we really count the amount of data on the internet? Let's establish a baseline - what constitutes "the internet" in terms of storage? Every website ever? What about apps and the data they create - do we include those databases because mobile apps use them? How many companies will volunteer how big those databases are? GoDaddy will probably be able to more-or-less say how much data they host, but how much of it is active data - does it have to be served up to count? Similarly, does this include Dropbox data that's technically accessible, but only to its end user? If so, what about end users who own their own Synology boxes and back up their pictures to it over the internet? Does the data on those home NAS units count? Do we limit protocols to HTTP, or are we also talking about FTP sites, NNTP servers (do we count the total amount of Usenet data, or does each company who peers that data count separately?), and data available via torrents? What about e-mail - does e-mail count if it's stored on a server and accessible via a web browser? What if it's only accessible via POP/IMAP?

      Even if *you* came up with a number that includes what you deem appropriate for '80% of the internet', it's not going to translate well. If your metric was "anything that is accessible from a computer and isn't behind a login prompt", that's going to be different than someone who says that Dropbox counts, which doesn't fit your criteria - undoubtedly petabytes of difference, making the measurement irrelevant.

  17. Re:Does is Run Linux? by Big+Hairy+Ian · · Score: 2

    It's a shame there are no baseline performance statistics it would be interesting to know how much of a game changer this thing really is.

    --

    Build a Man a Fire, and He'll Be Warm for a Day. Set a Man on Fire, and He'll Be Warm for the Rest of His Life.

  18. Re:"require a whole new kind of software" by St.Creed · · Score: 2

    Yes. I fondly remember the Transputer. Brilliant stuff, but noone wanted to learn Occam, one of the most elegant parallel-from-the-ground-up languages I know. But they invented parallellizing compilers and libraries for that. Suboptimal, but given the raw power of this beast, I'm not sure that matters much.

    --
    Therefore, by the (faulty) logic you're using, you're just a cow with a keyboard - osu-neko (2604)
  19. Re:Does is Run Linux? by ausekilis · · Score: 2

    Yea, but how many cat pictures do you need open at the same time?

  20. Re:Does is Run Linux? by David_Hart · · Score: 2

    Yea, but how many cat pictures do you need open at the same time?

    All of them, at once... obviously,,, You just can't have too many cat pictures...

  21. Re:For good or evil? by Hognoxious · · Score: 2

    That would make an awesome movie. Just the one, though.

    --
    Confucius say, "Find worm in apple - bad. Find half a worm - worse."
  22. Re:Does is Run Linux? by ausekilis · · Score: 2

    1 yottabyte = 2^80 or 10^24

    4096/1.44 = 2,844.4444

    So, basically 2,844,444,444,444,000,000,000,000,000 floppies.

    The weight of one floppy is 19g, in case anyone wants to do the conversion to VW Beetles.

  23. Re:Does is Run Linux? by edxwelch · · Score: 2

    Guess I need to buy another box