How Microsoft Rewrote Its C# Compiler in C# and Made It Open Source (medium.com)

Mads Torgersen, the lead designer of C# at Microsoft, remembers "Project Roslyn," which built an open-source, cross-platform compiler for C# and Visual Basic.NET "in the deepest darkness of last decade's corporate Microsoft: We would build a language engine! A unified, public API to C# code: We would redefine the meaning of "compiler". Of course, once you are building an API for the broad C# community, it is kind of a slam-dunk that it should be a .NET API, implemented in C#. So, the old dream of "bootstrapping" C# in C# was fulfilled almost as an accidental side benefit. Roslyn was thus born out of an openness mindset: sharing the inner workings of the C# language for the world to programmatically consume.

This in and of itself was a bit of a bold proposition in what was still a pervasively closed culture at Microsoft: We would share this intellectual property for free? We would empower tool builders that weren't us to better compete with us? The arguments that won the day for us here were about strengthening the ecosystem and becoming the best tooled language on the planet. They were about long-term growth of C# and .NET, versus short term monetization and protection of assets for Microsoft. So even without having mentioned open source, signing up for the cost and risk of the Roslyn project was a big and bold step for Microsoft....

F# released already in 2010 with an open source license and its own foundation -- the F# Software Foundation. The vibrant community that grew up around it soon became the envy of us all. Our team pushed strongly to have an open source production license for Roslyn, and finally a company-wide infrastructure emerged to make it real. By 2012, Microsoft had created Microsoft Open Tech; an organization specifically focused on open source projects. Roslyn moved under Microsoft Open Tech and officially became open source... C# language design and compiler implementation are now completely open processes, with lots of non-Microsoft participation, including whole language features being built by external contributors.
Torgersen's article says C# now enjoys "the scaling of effort via contribution of features and bug fixes, but also the insight and course correction we get through the instant, daily feedback loop that open source provides.

"It's been a long and wild journey, and one that to me is symbolic of the massive changes that Microsoft has undergone over the last decade."

Comments

Re:Opportunity wasted

[Anonymous Coward]

That's really dumb, whilst I agree there are too many languages out there, and people don't always need to make new ones when existing ones exist, we also don't want a monoculture where tech goes down one singular path.

The fact is C# and .NET were already well established, before LLVM even came into existence, and have a massive ecosystem around them with millions of devs and companies using it.

Thus, making it able to work for more people in more places is a good thing, and given it's strong base, like Java, it's a reasonable piece of alternative tech that should stay alternative and not be converged into a great LLVM monolith where progress will never happen because there's no competition anymore. That would be a terrible outcome for everyone, it's what happened when IE "won" the browser wars and we were all stuck with IE6 ruling the browser world for half a decade.

As for your comment on bugs, I'm not sure what your point is. You're basically arguing against open source altogether at that point because you're saying if a community finds and logs bugs it's a terrible thing? How fucking stupid are you? The whole point in open source is precisely so that many eyes can spot problems and collaborate to fix them. There are plenty of major open source projects that are nothing to do with Microsoft in the exact same position. If you criticise an organisation for properly tracking all it's bugs and doing so publicly and for allowing people to contribute more bug reports and also therefore allowing open source devs to also chip in and fix them, then you're basically arguing companies shouldn't open source because you'll shame them for letting other people work on the project.

People like you really need to get over your irrational Microsoft hate and take a leaf out of Torvald's book and go and get some professional help. You're exactly the sort of toxic open source community member he was talking about, because even when you've got literally everything you wanted from a formerly proprietary company - i.e. a complete open sourcing of one of their key products, you STILL bitch and moan at them and imply they shouldn't have done it. You're exactly the sort of person the OSS community doesn't need because you're exactly the sort of no-life vermin that pushes organisations and individuals away from contributing to it.

Re:Opportunity wasted

[Interfacer]

I thought the mindset of open source was that YOU did things how YOU wanted to do them, and provide the source so that people can fork if they disagree?
That is after all why there are over a thousand text editors for linux.
That is why you have X.org and X386 if I recall correctly.
VIM and emacs.
etc.

My guess is that no matter what Microsoft does, you will complain about it.

Re:Opportunity wasted

[Yaztromo]

That is why you have X.org and X386

The project was named XFree86. You were so very, very close :)

Yaz

Re:Opportunity wasted

[TheRaven64]

If they wanted to actually contribute to the community then they would have made an LLVM frontend

They did. LLVM is not currently a great fit for garbage collected languages, though the LLILC team worked with some of the Azure folks to improve it in this regard. LLVM also suffered from longer compile times (important for a JIT). The Roslyn architecture makes it easy to incrementally replace the JIT, because it supports trying to compile individual functions with a new JIT and falling back to the old one if it doesn't support all of the functionality required. The LLILC team made use of this when trying to bootstrap.

Re: How?

[phantomfive]

The answer in the article is that they wanted to and looked for a reason until they found one. The more interesting part is that is shows how deep the Windows development process is within Microsoft. I thought they would have learned their lesson with Vista, but nope. That's what stack ranking gets you, I guess.

Re: Opportunity wasted

[Billly Gates]

They have submitted code. Infact clang is an option in visual studio community as an option. .net core is available for Linux and so is visual studio code editor. Redhat is ditching Mono and even including Microsoft's Rosslyn and let's not forget WSL for Windows10 nor Android sdk including Android emulators with Visual studio either or the fact that Bill Gates himself owns a Samsung phone.

I am not saying to be a MS fanboy. What I am saying is MS is changing due to the world changing around them. It isn't 1999 anymore. Apple is much more powerful and popular than back then. Also IOS, Android, HTML 5 browsers, cloud computing, and a plethora of free languages and apis like python, R, rust, node.js, and others changed the landscape. Gone are VB, COM, vc++, Internet Explorer, and WinCE.

MS has a new CEO who realized they no longer set the pace of the industry and if they want to remain relevant they and not loose mellinial developers they need to include, not exclude.

So that is the argument. Remember Apple was cool here and open source too back in 1999 on slashdot. They turned assholish FAST and more aggressive than MS when they got power. All companies are the same once they corner a market

Re:First selfhosting copiler EVER!

[Anonymous Coward]

Letting your paranoia and politicalization of technology shows a person who has thrown away the concept of "using the best tool for the job". A good programmer is one who delivers the best solution not the one who willingly and reflexively ignores anything connected to MS. MS is their historically closed ecosystem with a more open and cross platform ecosystem. The closed ecosystem created one of the most successful corporations in history. The IT world has changed over the years and MS is changing what needs to be changed to remain a successful and profitable corporation. After all MS is not a charity. All the MS haters used the closed source and single platform positions to to anchor their criticisms. Now that MS is basically turning into a more open source and multi-platform stance what will people use to continue their crusade against anything related to MS?

.NET is still dangerous to depend on

[jbn-o]

[...T]he fact that one of Microsoft's main compilers is now working in the same way that gcc works (self hosting, open contribution, free software, open source) is amazing. In a very deep way, the free software foundation has won. Before they came along there was plenty of "open source" software but it was ignored and simply ripped off by companies like Microsoft who could incorporate open source innovation into their products without even having to give credit. Now Microsoft is openly releasing Open Source Software which is really also Free Software.

Here's one big difference: Microsoft's C# compiler (licensed under the Apache 2.0 license) is a way to get its users into becoming dependent on the patent-encumbered .NET (as quoted in the /. summary, the article said "Of course, once you are building an API for the broad C# community, it is kind of a slam-dunk that it should be a .NET API, implemented in C#."). Apache 2.0 is more recommendable than other permissive licenses but Apache 2.0 doesn't look out for your interests in modifying and distributing modified software as well as the GNU GPLv3 does. .NET is still a patent-encumbered trap. I can't say the same for GCC (licensed under the GNU GPLv3 or later with the GCC Runtime Library Exception.

A reminder of what limits you're taking on by using .NET:

The first limit is that you're only protected if you're distributing the code "as part of either a .NET Runtime or as part of any application designed to run on a .NET Runtime". So if you add any of the code to another project, then you lose protection and MS reserves the right to use their patents against you.

Secondly, the protection only applies to a "compliant implementation" of .NET. So if you want to remove some parts and make a streamlined framework for embedded devices, then your implementation won't be compliant and the protection doesn't apply to you.

This is very different from code licensed under the GNU GPLv3:

Code distributed under the GNU GPLv3, comes with a patent grant which basically says the contributors can't use their patents against the users for exercising the freedoms granted in the licence. [Therefore, Microsoft's] patent licence looks fine for users of the code published by Microsoft, but its protections disappear very quickly for those who wish to modify or re-use the code.

Re:Opportunity wasted

[xonen]

Do you know where those people work now? How do you feel about those companies?

Any long term /. reader will know that most big companies are frown upon. How Apple went from nerd's favorite to fanboys. How Google is almost synonym for Big Bro. How IBM went from markt leader to small boy.

Any big company should be observed with skepticism and reservation, not only the tech sector, but the tech sector simply has a big influence on our daily lives and the future to come. That's why they are under a microscope. It's also why they are not expected to change fast.

But when it comes to Microsoft it's also why 2 decades of bad behavior is not suddenly forgotten after one decade of being 'OS friendly' or 'no longer OS hostile', especially if you see how they treat their customers with win10 when it comes to privacy.

But any day Microsoft fanboys take over /. is a sad day, and today is one.