Slashdot Mirror

← Back to Stories (view on slashdot.org)

Microsoft Goes In For Hadoop

Posted by timothy on from the well-it-is-a-nice-name dept.

Frankie70 writes that after more than three years, Microsoft has "finally learned to stop worrying and love Hadoop." Frankie70 excerpts from the linked Wired article: "Any aversion to Hadoop disappeared on Wednesday, when the company announced that it will integrate the platform with future versions of its relational database, SQL Server, and its platform cloud, Windows Azure, an online service for hosting and readily scaling applications. The company is now working to port the Hadoop platform to Windows."

67 comments

  1. Hadoop is written in Java by binarylarry · · Score: 5, Funny

    So what they mean is, they're going to do a search and replace to make it compile as a C# application.

    --
    Mod me down, my New Earth Global Warmingist friends!
    1. Re:Hadoop is written in Java by Anonymous Coward · · Score: 0

      Not when there is IL that makes that port even easier.

    2. Re:Hadoop is written in Java by allenw · · Score: 1

      Actually, there is an ever increasing amount of JNI (read: C) code in Hadoop that is in the critical path for security and performance features. Most of that code is not very portable. So either MS is going to pay for some major overhauling of that code, completely new code/branch to replicate that functionality or MS Hadoop is going to be severely lacking in features/performance.

    3. Re:Hadoop is written in Java by shutdown+-p+now · · Score: 1

      If that C code is well-written, it shouldn't be hard to port it over - the porting would have to be done at the Java/C boundary, and .NET actually has it much simpler thanks to P/Invoke.

    4. Re:Hadoop is written in Java by allenw · · Score: 1

      It isn't. There is an incredible overuse of glibc/Linux-isms to the point that even porting it to another UNIX is difficult.

    5. Re:Hadoop is written in Java by shutdown+-p+now · · Score: 1

      Well then, perhaps those guys will clean it up architecturally while they're porting it, and submit the changes upstream.

    6. Re:Hadoop is written in Java by Ed+Avis · · Score: 1

      Since it's written in Java, they can just run it on the .NET virtual machine using IKVM.

      --
      -- Ed Avis ed@membled.com
  2. We know, we know... by Anonymous Coward · · Score: 0

    Embrace, extend, extinguish. Now, someone please say something insightful.

    1. Re:We know, we know... by Tsingi · · Score: 1

      How about Spiteful?
      Does taking the source from an Apache licensed product and converting it from java to .net mean you no longer have to observe the license?

      I'd say no, I wonder what M$ will say.

    2. Re:We know, we know... by MozeeToby · · Score: 1

      I have to wonder where the first line of the article came from. If it's from the author, nothing to see here, but if it's a near direct quote from the source it sounds more like a shoutout to Dr. Strangelove, which could indicate that they are still terrified of it but are embracing it because they don't feel they have any choice in the matter.

  3. Not wanting to put a dampener on things... by bernywork · · Score: 1

    I think MS getting involved with Open Source is great, but....

    We've seen the way that they work before, embrace and extend... This hasn't worked out that well for them before, but you have to ask if there is an alterior motive in there...

    --
    Curiosity was framed; ignorance killed the cat. -- Author unknown
    1. Re:Not wanting to put a dampener on things... by Anonymous Coward · · Score: 0

      There is no question that there is an ulterior motive here by Microsoft. Everything every person ever does is always done with such. The real question is will Microsoft's motives be in direct conflict with what is the overall good for the rest of the users/developers of Hadoop. With MS, that is how it almost always plays out so it would be pretty naive to assume otherwise in this case. My suggestion, short Hadoop when MS gets it in their clutches.

    2. Re:Not wanting to put a dampener on things... by Attila+Dimedici · · Score: 1

      I am not sure what an alterior motive is, but I am quite sure that MS has an ulterior motive for this. The only question is whether or not that ulterior motive is detrimental to the Open Source community.

      --
      The truth is that all men having power ought to be mistrusted. James Madison
    3. Re:Not wanting to put a dampener on things... by bernywork · · Score: 1

      > alterior

      bad spelling, ulterior. Sorry, my bad.

      --
      Curiosity was framed; ignorance killed the cat. -- Author unknown
    4. Re:Not wanting to put a dampener on things... by jbolden · · Score: 1

      They are quite publicly indicating their intention is to embrace and extend:

      1) Get Hadoop to work on Windows servers
      2) Create a Windows server management interface for Hadoop
      3) Create SQL Server extension to manage Hadoop.

      So we don't have to speculate, that's what they say they are doing. That being the case all that stuff might be useful for Hadoop.

    5. Re:Not wanting to put a dampener on things... by Forbman · · Score: 1

      Sell more Windows Server & SQL Server Enterprise/Data Center licenses?

    6. Re:Not wanting to put a dampener on things... by jbolden · · Score: 2

      I think they are telling the truth about their goals:

      1) Get Hadoop to work on Windows servers
      2) Create a Windows server management interface for Hadoop
      3) Create SQL Server extension to manage Hadoop.

      And the motive is:
      a) Sell server licenses
      b) Sell SQL Server licenses

    7. Re:Not wanting to put a dampener on things... by mandelbr0t · · Score: 1

      This is a smart move by MS. Microsoft is not working internally on any sort of NoSQL server, so they support an existing project that complements their own product. The very obvious goal is to integrate Hadoop with SQL Server management tools. The upshot is that Hadoop gets a leg up on their competitors (Yahoo! PNUTS, Google BigTable) and Microsoft sells more SQL server licenses. Seems to me to be a win/win.

      --
      "Please describe the scientific nature of the 'whammy'" - Agent Scully
    8. Re:Not wanting to put a dampener on things... by jimicus · · Score: 1

      You would be amazed how many people go nuts over the latest F/OSS platform du jour... and then complain that it runs first and foremost under Linux.

      Even if they're never going to go anywhere near the underlying OS anyway, still that gets brought up.

      Windows Server licensing is quite lucrative for Microsoft. So if they can now announce "Hadoop: Now certified for Windows (TM) Server" they can sell more licenses for Windows Server.

    9. Re:Not wanting to put a dampener on things... by aztracker1 · · Score: 1

      I think it's to integrate a map/reduce structure into SQL server... I haven't RTFA, but that is about it... I wouldn't necessarily expect them to use Hadoop directly, but to support Hadoop's interfaces. My $0.02 on this. I know a lot of people are using MongoDB, and other document centric datastores lately, and MS is moving to compete in their tool space. More power to them, doesn't mean it'll be my first choice.

      --
      Michael J. Ryan - tracker1.info
    10. Re:Not wanting to put a dampener on things... by cayenne8 · · Score: 0

      Sell more Windows Server & SQL Server Enterprise/Data Center licenses?

      What major data center (working with large volumes of critical data) in its right mind would ever even consider using MS SQL Server as its database? Who'd consider running their critical server for any database on a windows box?!?!

      Not in any major production player I've ever seen or worked at....

      --
      Light travels faster than sound. This is why some people appear bright until you hear them speak.........
    11. Re:Not wanting to put a dampener on things... by Anonymous Coward · · Score: 1

      NASQAQ uses sql server http://www.computerworld.com/s/article/106050/Microsoft_unwraps_flagship_database_SQL_Server_2005

      DirectEdge - 4th largest stock exchange uses sql server
      http://blogs.technet.com/b/dataplatforminsider/archive/2011/06/03/fourth-largest-us-stock-exchange-direct-edge-looks-to-sql-server-parallel-data-warehouse-for-big-data-needs.aspx

    12. Re:Not wanting to put a dampener on things... by allenw · · Score: 1

      This isn't about Microsfot getting involved with open source. This is about Microsoft not getting left out. Beyond the countless startups, Apache Hadoop already has major players like Amazon, Dell, EMC, HP, IBM, NetApp, Oracle, VMware, ... trying to make a dent in the community in some form or another. Hell, I have a SuperMicro catalog on my desk emblazoned with the Apache Hadoop logo all over it. Like Oracle, they are coming in very late to the party and now need to play catch-up. Buying off Hortonworks is a very fast way to do that.

    13. Re:Not wanting to put a dampener on things... by Thing+1 · · Score: 1

      I am not sure what an alterior motive is, but I am quite sure that MS has an ulterior motive for this.

      My ex-girlfriend was an ulterior decorator.

      --
      I feel fantastic, and I'm still alive.
  4. First... by knuthin · · Score: 1

    First they ignore you.
    Then they laugh at you.
    Then they port you on their platform.
    Then you win.

    Original: http://bit.ly/o3V3cA [Google Books]

    --
    Some apps are WYSIWYG. Some others are WYSIWTF.
    1. Re:First... by Anonymous Coward · · Score: 0

      Yeah, you win by... helping them!
      Go, FOSS, go!

    2. Re:First... by camperdave · · Score: 1

      First they ignore you.
      Then they laugh at you.
      Then they port you on their platform.
      Then you win.

      First they ignore you.
      Then they laugh at you.
      Then they port you on their platform.
      Then they add some convenient feature that they only make available on their platform
      Then they win

      FTFY

      --
      When our name is on the back of your car, we're behind you all the way!
    3. Re:First... by Anonymous Coward · · Score: 0

      First they ignore you.
      Then they laugh at you.
      Then they fight you.
      Then you win.

        -- Gandhi

      First they march you through hundreds of miles of jungle without food or water.
      Then they shoot you.
      Then they disembowel you.
      Then you lose.

        -- Gandhi, had the Japs won WW2

    4. Re:First... by knuthin · · Score: 1

      That was one another depressing possibility.

      Think I will go into a corner and cry for a minute now. :/

      --
      Some apps are WYSIWYG. Some others are WYSIWTF.
  5. Heh by Hatta · · Score: 4, Funny

    Someone should trick Timothy into reposting this article. Then he'd be duped into posting a dupe about hadoop.

    --
    Give me Classic Slashdot or give me death!
    1. Re:Heh by darthdavid · · Score: 1

      Someone should trick Timothy into reposting this article. Then he'd be duped into posting a dupe about hadoop.

    2. Re:Heh by Anonymous Coward · · Score: 0

      You must be new here.

    3. Re:Heh by Anonymous Coward · · Score: 1

      Congrats, you've demonstrated enough brain-power for kindergarten. If a kid stacks a blue lego block on top of another, you are able to imitate and stack a 3rd blue lego block.

    4. Re:Heh by Anonymous Coward · · Score: 0

      I lost my book about object oriented programming for hadoop.

      Got hadoopoop?

      No, I hadhadoopoop.

    5. Re:Heh by Gilmoure · · Score: 1

      *golf clap*

      --
      I drank what? -- Socrates
    6. Re:Heh by robot_love · · Score: 1

      And then I'd file a false complaint against it being a copyright violation in France.

      It would be a Hadopi dupe dupe Hadoop.

      But I digress...

      --
      .there is enough of everything for everyone.
    7. Re:Heh by Anomalyst · · Score: 1

      *golf clap*

      Something you get from being in the showers with Tiger Woods?

      --
      There is no right to feel safe thru security vaudeville at the expense of everyone's freedom, privacy and tax money.
  6. Embrace. Extend. Extinguish. by mrflash818 · · Score: 3, Interesting

    "Those that do not learn from history are doomed to repeat it."

    "Embrace, extend and extinguish,"[1] also known as "Embrace, extend and exterminate,"[2] is a phrase that the U.S. Department of Justice found[3] was used internally by Microsoft[4] to describe its strategy for entering product categories involving widely used standards, extending those standards with proprietary capabilities, and then using those differences to disadvantage its competitors.

    http://en.wikipedia.org/wiki/Embrace,_extend_and_extinguish

    --
    Uh, Linux geek since 1999.
    1. Re:Embrace. Extend. Extinguish. by mrflash818 · · Score: 1

      Microsoft allowed the pair to continue their contributions to the open source project, and Powerset, which was rolled into Redmond’s Bing search engine, continued to run atop Hadoop.

      This made Bing one of the first “shipping” Microsoft products to actually include open source code. But somewhere along the way, Microsoft moved the engine onto a proprietary platform...

      "Microsoft allowed the pair" -- Here

      "which was rolled into Redmond's Bing search engine" -- It

      "But somewhere along the way, Microsoft moved the engine _onto_a_proprietary_platform_" -- Comes *sigh*

      --
      Uh, Linux geek since 1999.
  7. as usual, the summary here is lacking by tomzyk · · Score: 4, Insightful

    its relational database, SQL Server, and its platform cloud, Windows Azure, an online service for hosting and readily scaling applications

    That's wonderful that the summary mentions what "SQL Server" and "Azure" are... but why no mention of wtf "Hadoop" is?
    Why do I need to RTFA just to find out what we're talking about here?

    Hadoop — an open source platform for crunching epic amounts of a data across an army of dirt-cheap servers

    --
    Karma: NaN
    1. Re:as usual, the summary here is lacking by slim · · Score: 1

      The Wired article tells you what Hadoop is, because it's written by journalists, and aimed at a broad readership.

      The /. summary tells you what SQL Server and Azure are, because the descriptions arbitrarily happen to occur in the Wired paragraph that's been quoted.

      The /. summary does not tell you what Hadoop is, because (unlike Wired readers), /. readers are expected to have some basic knowledge of the software world. If you think Hadoop is obscure, maybe this isn't the site for you?

    2. Re:as usual, the summary here is lacking by Anonymous Coward · · Score: 0

      Think of a search query where if you didnt get all the data back it is not that big of a deal. The term is eventual consistency.

      For something like facebook page updates, web searches, data averaging this makes some sense. For something where "i must get the the data or all my results are wrong" type of things (think my receipt out of a cash register) not so good.

      They are removing the C out of ACID to achieve speed.

    3. Re:as usual, the summary here is lacking by camperdave · · Score: 1

      Me too! I had to double click on Hadoop, then right click and choose 'Search Google for "Hadoop"' from the context menu. Then I had to switch tabs and scan for the wikipedia site, click on that, and then skim through the wiki for a few seconds just to figure out whether it was worth it to read anything beyond the summary.

      Sadly, this is happening more and more with Slashdot.

      --
      When our name is on the back of your car, we're behind you all the way!
    4. Re:as usual, the summary here is lacking by Anonymous Coward · · Score: 1

      I'm with the OP. Quoting an article is fine, even with descriptions one should know... but if an article is primarily about something that isn't common, then a nice little blurb about it would save AT LEAST TWO PEOPLE a wiki call. Hadoop isn't as ubiquitous as you believe. Coming from a heavy consulting background in coding, I've never seen this.

      Oddly, by your logic, /. readers don't know what SQL Server and Azure are. I'll put dollars to doughnuts that more people USE SQL Server than KNOW what Hadoop is. Looking at the WIKI, I'm also willing to bet that you aren't pronouncing Hadoop correctly (not unlike the original GIF pronunciation).

    5. Re:as usual, the summary here is lacking by CannonballHead · · Score: 1

      The /. summary tells you what SQL Server and Azure are, because the descriptions arbitrarily happen to occur in the Wired paragraph that's been quoted.

      So what you're saying is .... the person who wrote the summary doesn't know how to write a good summary or pick a good quote. :) (no, I'm not new here...)

    6. Re:as usual, the summary here is lacking by slim · · Score: 1

      Oddly, by your logic, /. readers don't know what SQL Server and Azure are.

      Nope. As I said, the definitions for those arbitrarily happen to be in the quote chosen for the summary. But those definitions aren't the pertinent part of the quote.

  8. Dont look behind that curtian! by Lumpy · · Score: 1

    "the company plans to eventually release its work back to the open source community."

    That is a bit too vague... because 100 years from now is "eventually"

    --
    Do not look at laser with remaining good eye.
    1. Re:Dont look behind that curtian! by jbolden · · Score: 2

      If you look at their todo list they could release it constantly it won't matter. What they are doing is essentially creating extensions for their commercial products that work with Hadoop. I think they have every intention of trying to get the small parts that need to be in Hadoop back into the main tree.

  9. ... Last by Anonymous Coward · · Score: 0

    Then they embrace you.
    Then they extend you.
    Last they extinguish you.

  10. Amazing! by Anonymous Coward · · Score: 0

    That's fantastic! ...I'd be even more excited if there had been a 1 liner about what in the allspark Hadoop is...

  11. Really surprise? by Anonymous Coward · · Score: 0

    Half a year ago Microsoft was headhunting Hadoop developers from all around the world, and this fact was pretty well known in the Hadoop community. Why such a surprise now?

  12. One issue that bothers me is the license... by Anonymous Coward · · Score: 0

    I don't trust big for-profit software companies. If they are involved, it's not good for the OSS/free/libre community other than to say "Look! Guess who is using our software!". Big for-profit companies generally do what is good for them and their bottom line. For once, I wish OSS/free/libre software authors would really look at what's behind a license before they select a license. I don't want certain entities to profit from open source when their actual goals go against the OSS/free/libre community. GPL3 is the way to go. Flame on should you wish, but I've been in this industry far too long to not care about the long-term implications of getting in bed with companies who say one thing and do another. Embrace and extend are the watchwords here. Just goes to show that in the end, OSS is always better and they use it because they can do no better. Let them use it, but keep them under the watchful eye of the GPL.

  13. Dryad by Utopia · · Score: 1

    Microsoft has a equivalent to Hadoop known as Dryad.
    They should have open-sourced Dryad a long time ago.

    I wonder what is going to happen to Dryad with this focus on Hadoop.

    1. Re:Dryad by mandelbr0t · · Score: 3, Informative

      Dryad is not quite Hadoop. From their whitepaper:

      We can map the whole relational algebra on top of Dryad, however Dryad is not a database engine: it does not include a query planner or optimizer; the system has no concept of data schemas or indices; and Dryad does not support transactions or logs

      I can see how Hadoop would supplement their own research in this field.

      --
      "Please describe the scientific nature of the 'whammy'" - Agent Scully
  14. Denver Open Source Group meets at MicroSoft by peter303 · · Score: 1

    In the beginning I thought it was ironic. But its very generous of them to provide a free meeting room to our open-source computer study group. And MSFT people attend, but dont speak often.

    1. Re:Denver Open Source Group meets at MicroSoft by Anonymous Coward · · Score: 0

      This is the same thing that happened to our mobile group here the east, where MSFT sponsored the events, but when time and time again, no one is actually developing on their platform, MSFT drop the sponsorship.

      MSFT is here, because customer ask them to, not because they want it, it's all for short term results. MSFT is in no way committed to open-source, it's still a lip-service to me.

  15. test from my kindle by Anonymous Coward · · Score: 0

    tsting.....

  16. So... by Anonymous Coward · · Score: 0

    Does this mean MS will somehow kill Hadoop? Cause that'd be great.

  17. Microsoft Goes Hadooken by Spy+Handler · · Score: 1

    that's what i thought it said...... MS going all street fighter

  18. Re:oh gimme a f______g break by Anonymous Coward · · Score: 0

    Did you think that you could hide from us by going 100% FOSS, freetard? Muhahahahaaha... we are here to SPREAD EVIL and MAKE YOU SUFFER. If we have to write FOSS for that, we will do so. Resistance is futile. You will be mentally eviscerated.

  19. EMC Greenplum HD? by Anonymous Coward · · Score: 0

    Admittedly it's Linux based, but the full price Greenplum HD runs on Hadoop ported to C. So it not only can be done, it can be done well.

  20. The cool parts by davevr · · Score: 1

    The two cool parts of this announcement:

    1) They are contributing the bits needed to make it work on windows back to open source (Hortonworks is helping that make sure that goes smoothly)
    2) They are making JavaScript a first-tier language for writing map/reduce jobs, and contributing THAT work back to the community.

    That is awesome.