Slashdot Mirror

← Back to Stories (view on slashdot.org)

VMware's Serengeti Brings Hadoop To Virtual, Cloud Environments

Posted by ryuzaki0 on from the lions-to-be-added-in-future-patch dept.

Nerval's Lobster writes "VMware's Serengeti is a new open-source project for deploying Apache Hadoop in virtual and cloud environments. Serengeti 0.5 is available as a free download under the Apache 2.0 license. It has been designed as distro-neutral, with support for Apache 1.0, CDH3, Hortonworks 1.0 and Greenplum HD 1.0. Of course, VMware isn't the only company seeking to leverage the increased interest in Hadoop. In June alone, midsize IT vendors such as Datameer, Karmasphere, and Hortonworks have all announced platforms that utilize the framework in some way. Research firm IDC recently predicted that worldwide revenues from Hadoop and MapReduce will hit $812.8 million in 2016, up from $77 million in 2011."

6 of 28 comments (clear)

  1. I have to admit... by busyqth · · Score: 3, Funny

    So, if I've got only one server, then up to now, I would have to just run an application on that server.
    But now, with only a little overhead, I can pretend to be running the same application in a distributed manner on a cluster, even though it's actually still running on the single server.
    I have to admit this is pretty awesome.

    1. Re:I have to admit... by ArsonSmith · · Score: 2

      Or you have a smaller cluster of big boxes running a big cluster of smaller boxes(vms)

      --
      Paying taxes to buy civilization is like paying a hooker to buy love.
    2. Re:I have to admit... by abigor · · Score: 3, Interesting

      Yes, of course you can manually set up Hadoop in whatever environment, but it's a pain and generally speaking management is annoying. This new project appears to alleviate at least some of that, making it easy to remotely deploy and manage a Hadoop cluster. At least, that's what I got from the demo video - there's probably more to it.

      Regarding Hadoop, I'm always surprised by its popularity given the relative fragility of HDFS (the NameNode is a single point of failure; other distributed filesystems have beaten this problem) and the dubious, beta-like quality of the tools built on top of it (Pig, etc.)

  2. Shiny - High Revenue by Lord+Grey · · Score: 2

    From TFS:

    Research firm IDC recently predicted that worldwide revenues from Hadoop and MapReduce will hit $812.8 million in 2016, up from $77 million in 2011.

    Notice that the revenue is directed toward the few companies supporting and extending Hadoop. If you're working for one of those companies, congratulations. If you're working for one of the companies that is spending its money on this new shiny thing, you're probably in for a ride (one way or another). The technology is definitely good, I'll grant you that. But it is not the solution (or, not a very good solution) for many of the problems IT/data shops have. It really seems that a lot of people are jumping on the Hadoop bandwagon because "everyone else is getting it" and not because it will solve particular, concrete, existing problems. Or, it will solve exactly one relatively small, concrete, existing problem while erecting a complex infrastructure that must be supported for several years, making it more of a PITA than a solution.

    Anyway, back to my original point: I think this revenue citation is more of an indication of a technology bubble and successful marketing than anything else. The price IT will pay for that bubble will probably far exceed the original cost.

    --
    // Beyond Here Lie Dragons
    1. Re:Shiny - High Revenue by Rakishi · · Score: 3, Interesting

      As someone who actually uses Hadoop, you're so far off the mark you've hit a bystander in the head. Dealing with large amount of data is a major PITA. If you don't understand that then you must never have worked with anything but trivial data sets. Hadoop fixes much of it, period. Without having to spend insane amount of money on databases, DBAs and still not being able to scale properly. It's not optimal but it works, it scales and it's flexible.

      That's why companies are moving to it.

    2. Re:Shiny - High Revenue by Rakishi · · Score: 3, Insightful

      Really, so what would you use to deal with 10Petabytes of data while going through 1Petabyte of it per day? I'm sure Google, Ebay, LinkedIn, Twitter, Yahoo and Facebook have no idea what they're doing. Hahaha. Get back to me when you're running a multi-billion dollar company.

      That just indicates you lack the skills and knowledge to deal with big data sets.

      Keep thinking that, I'm going to be over here enjoying my guaranteed job security with occasional breaks to beat hordes of recruiters away. No, seriously, keep thinking it, more idiots like you out there the better my job security will be.

      See, I've actually got experience dealing with big data which I'm guessing is a lot more than you can say. I've talked to companies that think like you, it's downright hilarious to watch their jaws drop when you casually mention how trivial going through data is for us on Hadoop. A month long data project for them is a ten minute query for me that I let run for a few hours. I've personally played in both worlds, you can keep the alternative while I get actual work done.