Slashdot Mirror

← Back to Stories (view on slashdot.org)

Microsoft To Open Source Cloud Framework Behind Halo 4 Services

Posted by timothy on from the part-of-a-continuing-series dept.

angry tapir writes Microsoft plans to open-source the framework that helps developers of cloud services like those behind Halo 4. Project Orleans is a framework built by the eXtreme Computing Group at Microsoft Research using .NET, designed so developers who aren't distributed systems experts can build cloud services that scale to cope with high demand and still keep high performance. The Orleans framework was used to build several services on Azure, including services that are part of Halo 4. The code will be released under an MIT license on GitHub early next year.

50 comments

  1. Halo MCC by Anonymous Coward · · Score: 1

    Should've open sourced it before Halo MCC was released. Would've saved Microsoft a whole lot of headaches.

    1. Re:Halo MCC by Anonymous Coward · · Score: 0

      Halo MCC problems weren't with processing load or scaling, they were poor network code and dodgy matchmaking code.

    2. Re:Halo MCC by Anonymous Coward · · Score: 0

      Thats basically what the Orleans framework does for Halo4 is the matchmaking.

      Is a similar type of framework as Akka with a few fundamental differences.

      I'm glad MS are finally embracing open source and releasing things like this, even if it has had a bumpy past.

    3. Re:Halo MCC by ncmusic · · Score: 1

      Thats basically what the Orleans framework does for Halo4 is the matchmaking.

      This isn't true

  2. and whats a good use of this by Anonymous Coward · · Score: 0

    outside of games, where you have very small amounts of data to shove around quickly

    1. Re:and whats a good use of this by Anonymous Coward · · Score: 0

      Trading, maybe?

  3. First they... by sydsavage · · Score: 1

    Ghandi quote in 3... 2... 1...

    1. Re:First they... by Guspaz · · Score: 4, Funny

      "Our words are backed with NUCLEAR WEAPONS!"
      - Ghandi

    2. Re:First they... by binarylarry · · Score: 1

      Boom Bitch!

      --
      Mod me down, my New Earth Global Warmingist friends!
    3. Re:First they... by s.petry · · Score: 1

      Is that NUCLEAR or NUCULAR?

      --

      -The wise argue that there are few absolutes, the fool argues that there are no probabilities.

  4. please keep closed! by Masked+Coward · · Score: 2

    Whatever it is that made Halo 4 (cloud-based or otherwise) should remain closed. Or better yet, incinerate it.

    1. Re:please keep closed! by grcumb · · Score: 1

      Whatever it is that made Halo 4 (cloud-based or otherwise) should remain closed. Or better yet, incinerate it.

      Agreed. 'Software that makes it easy for non-experts to do expert tasks' will one day be recognised for its role in causing the downfall of civilisation as we know it. By then, of course, it will be too late.

      Some among you may think that's overstating things. Some among you are also .NET developers, so what do you know?

      Seriously, though: From the Airbus crash to high frequency trading to the Sony hack, examples abound of how enabling and empowering mediocrity is the first ingredient of every modern tragedy.

      --
      Crumb's Corollary: Never bring a knife to a bun fight.
    2. Re:please keep closed! by Anonymous Coward · · Score: 0

      I disagree. Encapsulation and abstraction of complexity is natural and humans are great at breaking complexity apart and making the common-man able to accomplish what was one impossible. The high performing scalable systems problems will be abstracted to such a degree that journeymen will be able to practically point and click there way through this problem. For you, it's time to learn the next big thing after ubiquitous scale. If you sit still the "idiots" catch up with you and you become one of them.

    3. Re:please keep closed! by donaldm · · Score: 1

      Whatever it is that made Halo 4 (cloud-based or otherwise) should remain closed. Or better yet, incinerate it.

      Well it is designed to run on .NET which is open source as well but licensed under Reasonable_and_Non_Discriminatory_Licensing which if you read this seems like a minefield full of flowers since Microsoft holds lots of patents on the all over infrastructure. Basically this will be a Microsoft only thing. Other companies enter at their peril. :)

      --
      There ain't no such thing as proprietary standards only proprietary formats. Standards are by definition open.
    4. Re:please keep closed! by Shados · · Score: 1

      .NET is slowly being "re-opensourced" under MIT license though.

    5. Re:please keep closed! by Opportunist · · Score: 4, Insightful

      If it didn't lead to cargo cult programming I would maybe even agree. Sadly, that's exactly where we're heading. What encapsulation and abstraction SHOULD bring us is less reinventing of the wheel and more focus on the task at hand. What it DOES get us is more and more people who create software we depend upon who don't have the foggiest idea of just what they're doing.

      This is actually an issue. And I'm not even talking about when you ask someone who claims to be a programmer why he chose this over that sort algo and he looks at you blankly, asking "what's the difference"? Or, my personal favorite, "'cause that one used one parameter less so it has to be faster".

      --
      We used to have a Bill of Rights. Now, with the rights gone, all we have left is the bill.
    6. Re:please keep closed! by Opportunist · · Score: 0

      Wake me when they're done. 'til then, keep that 10 foot pole ready.

      --
      We used to have a Bill of Rights. Now, with the rights gone, all we have left is the bill.
    7. Re:please keep closed! by grcumb · · Score: 0

      I disagree. Encapsulation and abstraction of complexity is natural and humans are great at breaking complexity apart and making the common-man able to accomplish what was one impossible.

      No dispute there. The problem, though, is not that we make easy things simple and hard things possible (pace, Larry Wall). It's that we have of late developed a tendency to simplify too far. Microsoft is famous for making systems administration and certain types of programming 'click-and-drool' easy. And hyperbole aside, the cost to society of the half-competent people who found gainful employment due to this charade can be measured in the many billions.

      You're absolutely right in that commercial flying is safer than ever, notwithstanding the tendency in airlines to pressure senior pilots out in favour of cheaper, younger staff. And those working in HFT would likely be wreaking havoc by other means if they didn't have software and fibre-optics to enable them. I guess my tongue hadn't entirely left my cheek when I wrote that last para.

      BUT... Microsoft has contributed significantly to a general downward trend in the quality of software and systems integrity. And they've done so by marketing the idea that with the right tools, tool users can be commoditised. And that really, really sucks.

      --
      Crumb's Corollary: Never bring a knife to a bun fight.
    8. Re:please keep closed! by Dutch+Gun · · Score: 2

      Well, yes, there are plenty of crappy programmers out there. And some developers always are looking for ways to make things simpler for everyone by creating all-encompassing frameworks, but in the end, almost invariably, it does lead to a lot of cargo cult programming practices with an over-reliance on frameworks and little understanding of how things work internally.

      That being said, there's nothing wrong with using higher-level systems and frameworks so long as you understand the tradeoffs you're making. I'm an expert C++ programmer (videogame development), but I always turn to C# / .NET for internal tools, because I'm so much more productive in that language. The tradeoff, of course, is that the tools tend to be much less efficient, as well as some extra work for writing interop with native code. It's not as much of an issue since I can assume we have more powerful development machines using those tools, and high-speed is less critical than efficient workflow, functionality, and safety.

      I think it's the same sort of tradeoff you'd be making when using this sort of framework. It's never going to be as optimal as a carefully tuned, custom crafted solution. As such, for extremely large or complex systems, or anything that absolutely demands extreme efficiency, you'd be better off with custom solutions written by experts. However, for projects with more modest goals, it could be that the requirements in productivity outweigh the costs of a custom, low-level solution. As long as the decision is made knowing these tradeoffs, it's fine.

      The problem is that the vendor never tells us about these tradeoffs. For years, Microsoft touted C# / .NET as the next big thing, and told us that "pretty soon" we were going to see performance comparable to native code. It never happened. Natively compiled C++ still kicks the crap out of managed code in real world scenarios. Most major client-side applications are still written in native code. I suspect it will be similar for this sort of platform. Yes, it will work, but it's never going to be nearly as optimal or flexible as a custom solution expertly designed and optimized for the particular task at hand.

      --
      Irony: Agile development has too much intertia to be abandoned now.
    9. Re:please keep closed! by Anonymous Coward · · Score: 0

      Until they do that, I won't even touch it with that...

    10. Re:please keep closed! by Anonymous Coward · · Score: 0

      At least we know it will work and not have any security holes because Open Source yada yada.

    11. Re:please keep closed! by Anonymous Coward · · Score: 0

      And hyperbole aside, the cost to society of the half-competent people who found gainful employment due to this charade can be measured in the many billions.

      I would certainly like to see how you are measuring this.

      BUT... Microsoft has contributed significantly to a general downward trend in the quality of software and systems integrity.

      Citation? These days systems are more complex yet have more uptime than ever, even Microsoft systems that were buggy with frequent blue screens in the 90s are rock solid these days. Vulnerabilities exist across the entire range of software both proprietary and free but the worst of attacks these days are in social engineering rather than exploiting software.

      The general trend certainly seems to be upward as systems are far more complex and far more stable these days.

    12. Re:please keep closed! by hairyfeet · · Score: 1

      But with the exception of games and maybe HFT is there any reason we should give a shit? Seriously computers are soooo insanely overpowered now it isn't funny so if this language is 4% slower than that language but is a hell of a lot easier to work with who cares? We got more cycles than we know what to do with anyway!

      --
      ACs don't waste your time replying, your posts are never seen by me.
    13. Re:please keep closed! by Dutch+Gun · · Score: 1

      From someone who uses both C++ and C# on a regular basis, my experience has been that the difference is fairly significant, not just 4%. That aside... you ask who cares? Examples:

      Demanding Applications
      If your app is extremely large, complex, or graphically intensive, you can probably benefit from a native performance boost. There's a reason office suites and graphics programs are written in C or C++. Games, of course, fall into this category as well.

      Simulations
      In scientific simulations, there's no such thing as "fast enough". These guys still require supercomputers on occasion, and you can bet they're concerned with the efficiency of their code, since they typically have to rent time on them.

      Small-form Devices
      For more computationally constrained platforms, such as in tablets, phones, and now even watches or other in-home smart devices benefit from improved speed and tighter control of memory. Also, keep in mind that lower CPU usage means more efficient battery use, which is critical for many small devices.

      Large-scale Server Applications
      For server-side applications, run-time efficiency can actually trump programmer productivity in importance when scaling up to very large numbers of users, like Google, Facebook, Microsoft, or Amazon has to do. A 10% increase in efficiency leads directly to a non-trivial savings in power costs of your server farm, so in these scenarios, efficiency can be very important. Of course, when Microsoft is renting you it's servers, it's more than happy to have you use C#, since you're the one paying for the servers. My bet is that their own command and control code is completely native.

      Today's programming world is especially diverse - much more so than it used to be. Not everyone is writing business apps for desktop PCs.

      --
      Irony: Agile development has too much intertia to be abandoned now.
    14. Re:please keep closed! by Anonymous Coward · · Score: 0

      The problem isn't performance. It's high-level languages lowering the barrier of entry to software development, which means the hoi-polloi can start writing code, which of course the beardlords hate because they get off on being gatekeepers

      God forbid a developer doesn't know how sort() works under the hood, The reason the algo that uses one less parameter is faster is because it's one less thing to have to remember.

    15. Re:please keep closed! by gbjbaanb · · Score: 0

      only the shitty stuff they don't care about - all the good code is kept locked away, for good reason. Its only the crap that isn't so bad they don't want you to look at, but they do want you to maintain that they open source.

      IIRC all the cloud goodness they have went C++ a while back, as they looked at how much it cost them to run .NET code on all their servers, when you have a million servers the loss in efficiency of running .NET everywhere adds up to a significant amount. Hence their "C++ Renaissance" of a few years back.

    16. Re:please keep closed! by gbjbaanb · · Score: 1

      My bet is that their own command and control code is completely native.

      mine too. See Herb Sutter's speech at Build 2011, (Here's a transcript)

      What happens in the Datacenter? Citing a study, Mr. Sutter claims that the biggest cost is Hardware and Power. These account for 88% of the total cost of running a Datacenter. Performance/Watt has a direct impact on this, and it can be improved by having more efficient programs.

      and

      Quoting Bjarne Stroustrup: âoeMy biggest contribution to the fight against global warming is C++â(TM)s efficiency: Just think if Google had to have twice as many server farms! Each uses as much energy as a small town. And itâ(TM)s not just a factor of two⦠Efficiency is not just running fast or running bigger programs, itâ(TM)s also running using less resources.â

      So they're thinking of their dollars - more efficient programs means fewer servers for the same workload, means less cost and bigger bonuses for the bosses.

      This is why Microsoft picked up C++ again after practically killing it off, or at least keeping it around as a niche product solely for the Windows and Office teams that still used it. Its also why the dev team are going all-in on the native .NET, using the C++ backend compiler to create native code from .NET programs, which will help but won;t solve the fundamental efficiency problems of .NET like GC that effectively encourages inefficient use of memory.

    17. Re: please keep closed! by Billly+Gates · · Score: 1

      This is cool for a project I am working on. I plan to see if I can create a .com that will do business transactions. I need no latency but at the same time require ACID to ensure each transaction will be written to disk :-(

      My Idea is to have no sql and sql databases where something like this will do the transactions to ACID.

      --
      http://saveie6.com/
    18. Re:please keep closed! by DuckDodgers · · Score: 1

      The problem is, most of us are crappy programmers for a few years, or even more than a decade, before we're decent software developers. So our pain is inevitable - if you stop the underqualified novices from doing work, you've saved yourself some pain but you've reduced the number of developers of the future.

      The ideal solution, as far as I can tell, is more mentorship and closer supervision of novices. You let the same learners write the same garbage code, but then someone who knows what they're doing and is specifically assigned to spend half of their time watching the novices will review and reject most of their work - while providing an explanation as to why it was rejected and what should be done differently.

      But in reality, most companies are in such a rush to get the next release to market that they let the novices work unsupervised and put the senior developers at work in other areas. Then two months or six months or three years later the novices have moved on, and the senior developers are too busy dealing with technical debt from the previous novices to work on new features or shepherd the latest round of novices.

      But if I had been blocked from writing code when my code was crap - and my code was crap for a long time - I'd never have reached the point where I'm at today. I'm no genius, I'm no industry leader, but I can recognize most of the errors I made five and ten years ago.

    19. Re:please keep closed! by Anonymous Coward · · Score: 0

      http://microsoft.github.io/

      CoreFX, the future of .NET, MIT License:
      https://github.com/dotnet/corefx/blob/master/LICENSE

      ReferenceSource, the past of .NET, MIT License:
      https://github.com/Microsoft/referencesource/blob/master/LICENSE.txt

    20. Re:please keep closed! by Dutch+Gun · · Score: 1

      I'm guessing that's where I originally picked this information up, but I couldn't remember the exact source, so I didn't cite it. Thank you for the reminder and the links to those sources. I also recall one of Bjarne's talks discussing another reason why C++ performs better than managed languages, and that's because of better cache coherency, which is a pretty crucial for modern processors.

      Anyhow, as someone who relies on both C++ and C#, I'm glad to see them both moving forward with solid support from MS.

      --
      Irony: Agile development has too much intertia to be abandoned now.
    21. Re:please keep closed! by Opportunist · · Score: 1

      The problem is not inefficient coding style. The problem is that people not knowing what they're (really) doing leads to security issues. If a person does not know WHY he is supposed to do something and just does it because he's told to do it, he does not know what to look for when trying to avoid pitfalls.

      Look at the OWASP Top 10. And while I don't think too highly of them as an organization, their top10 reflect pretty well what problems I find in web apps during audits myself. Even though it looks almost unbelievable that SQL injections are STILL the leading cause of web insecurities. Despite there being a near foolproof silver bullet for this problem (parameterized queries and prepared statements, if used correctly). There is exactly NO reason why (with programmers deserving that name) this should even still be on the list. Let alone at the top spot.

      Of course people use frameworks and languages that allow abstraction. Actually this can well increase security, nobody in their sane mind would write their own security libraries. And while some might now cry that OpenSSL is maybe not the best example with heartbleed and POODLE still well in our memory, you do NOT want to see the security holes the average roll-your-own security implementation has. Using libraries is a GOOD thing.

      But the person using it has to know what they're doing! Just copy/pasting something off stackexchange is NOT going to end up in decent code!

      --
      We used to have a Bill of Rights. Now, with the rights gone, all we have left is the bill.
    22. Re:please keep closed! by DuckDodgers · · Score: 1

      But unless you're doing something that's waiting for disk input and output or network input and output, the difference isn't 4%. The micro-benchmarks at the Great Computer Language Shootout page are a simple starting point. For Java vs. C++, for some reason the chart (section 2) doesn't show memory difference correctly. The measurements section (section 3) does - Java is about 2x slower than C++ in most of the benchmarks, but only comes within 2x of C++ for memory usage in a few places, in most other micro benchmarks the memory difference is higher, even 20 times higher. ( http://benchmarksgame.alioth.d... )

    23. Re:please keep closed! by gbjbaanb · · Score: 1

      true, same here - I just wish they'd made the distinction between then better - C# for RAD tooling, C++ for performance/efficiency/"heavy lifting" like it used to be with VB and C++ but instead it seems C# was designed to be "the new Microsoft language" by one division so they wouldn't have to work with the other division :-(

      If you want to see a better benchmark for performance, look at MSDN magazine's article on WWS, where the Windows division wrote a compatible version of WCF using C so you can see a good comparison of 2 things that do *exactly* the same thing, but written differently.

    24. Re:please keep closed! by im_thatoneguy · · Score: 1

      Also most internal tools aren't CPU bound it's either Database or Disk bound anyway. A lot of internal tools at media and corporate offices is just a workflow enhancer.

    25. Re:please keep closed! by Anonymous Coward · · Score: 0

      But if I had been blocked from writing code when my code was crap - and my code was crap for a long time - I'd never have reached the point where I'm at today. I'm no genius, I'm no industry leader, but I can recognize most of the errors I made five and ten years ago.

      Those of us that are geniuses have gone through the same process.

      It's like any other art. You won't always be able to learn from other's mistakes, though we all build on the work of others to some extent (sometimes without realizing it). You can't learn from your own mistakes until after making them. You'll make some mistakes for years before realizing that you were making them. You won't learn how to correct some mistakes for years, even though you realize or at least suspect you're making one. And so forth ...

      Those that tell you otherwise are lying or idiots.

  5. Stop the press! by Anonymous Coward · · Score: 0

    I will forever remember where I was and what I was doing when I read about this momentous news item.

  6. Open Source Vaporware by salesgeek · · Score: 0

    I'm glad that Microsoft is releasing this. I really am. But you don't have to do the whole vaporware release where you say "We'll be releasing this awesome thing early next year." Just release the code and announce it when you do. Old habits die hard ...

    --
    -- $G
  7. What is this Codeplex crap? by Anonymous Coward · · Score: 0

    What is this Codeplex crap? On OSX + Chrome, viewing any of the source files causes the file contents to blank out when I scroll down the page. Here's one example:

    http://orleans.codeplex.com/SourceControl/latest#src/samples/AzureWebSample/WebRole/WebRole.cs

  8. Encouragement... by FearTheDonut · · Score: 4, Insightful

    We've been asking for years now (decades, even) for Microsoft to become more open. Regardless of their motivation, this kind of behavior should be encouraged, rather than ridiculed. To my knowledge, none of the other platforms they've open sourced has 'taken back' by them, as some conspiracy theorists have anticipated. While I'm under no illusion that Office or Windows will ever be open sourced, I'm very happy that much of their other platforms are becoming more open and hope they continue to do so.

    1. Re:Encouragement... by drinkypoo · · Score: 1

      Show me a license which protects my interests, and I'll show you some enthusiasm.

      --
      "You're right," Fisheye says. "I should have set it on 'whip' or 'chop.'"
    2. Re:Encouragement... by Anonymous Coward · · Score: 0

      >The code will be released by Microsoft Research under an MIT license and published on GitHub.
      http://en.wikipedia.org/wiki/MIT_License

      Now make with the enthusiasm, work today has drained my batteries.

  9. Exit strategy? by RobSwider · · Score: 1

    Does this give Microsoft a legitimate exit strategy for when they decide they're done supporting Halo 4 Multiplayer? Can they just say "You can still play online with friends!**"? Will they start handing over the keys, as it were, sooner than they ordinarily would because "someone will pick up the slack"? End-of-life is becoming an increasingly important issue with so many games going all-or-nearly-all online.

  10. Do or die time by funkymonkjay · · Score: 1

    It's do or die time for MS's .Net and Azure so they need to throw down any and all incentive to get people to get on board. I applaud the new CEO's aggressive switch over to open source but it's a little too late. That moment has come and gone. My previous work place experienced this first hand. Potential buyers loved the product but not the platform that it was written in. They wanted linux/java, not MS/c++/c#.

  11. Github by Anonymous Coward · · Score: 0

    Sorry but unless someone posts it somewhere else I won't check out the code directly from github.
    Github kept until recently a list of 11000 twitter handles labeling people as misogynysts with no trial.
    If opensource can't be apolitical, github can keep their free services for themselves.

  12. Not sure it's worth it by trev.norris · · Score: 1

    I played Halo 4 for over 400 hours, and I'm currently playing MCC exclusively. While I love the game, I can also say that the multiplayer in Halo 4 and definitely now in MCC are definitely lacking. So I'm not sure Microsoft is doing the world a favor by releasing this. What I can hope is that users will be able to help Microsoft make Halo online gaming experience better.