Microsoft Starts Working On an LLVM-Based Compiler For .NET

An anonymous reader writes Are the days of Microsoft's proprietary compiler over? Microsoft has announced they've started work on a new .NET compiler using LLVM and targets their CoreCLR — any C# program written for the .NET core class libraries can now run on any OS where CoreCLR and LLVM are supported. Right now the compiler only supports JIT compilation but AOT is being worked on along with other features. The new Microsoft LLVM compiler is called LLILC and is MIT-licensed.

Comments

Developers, Developer, Developers

[CSHARP123]

The old CEO was shouting developers, developers and pushing them aside. This CEO doesn't make a peep but quietly bringing in the developers

Re:Developers, Developer, Developers

[Richard_at_work]

How did Ballmer push developers aside? Under the latter part of his reign, Microsoft started open sourcing a lot of their developer frameworks etc (ASP.Net MVC in 2012, Entity Framework in 2013 etc) and we saw fairly large shifts in developer conferences and support.

Re:Developers, Developer, Developers

[ranton]

Well I can say these announcements have brought me back to developing in .Net. C# has been my favorite language by far ever since v3.5 (and was my favorite by a little since v2.0) but its vendor lock was becoming too much of a liability.

The moment I can go back to C# and easily have my code run on *nix servers I will drop Java in a heartbeat. Just being able to use LINQ again in my professional life will be a blessing. And going back to Visual Studio over Eclipse / Netbeans / and even IntelliJ is also something I have been longing for.

Re:Developers, Developer, Developers

[neilo_1701D]

How did Ballmer push developers aside?

He pushed them aside by killing development systems (VB6,FoxPro), introducing new technologies with the promise of how critical they were to the future (Silverlight), and allowing the crown jewels of Windows development, the Win32 API, to slowly become more irrelevant with endless layers of cruft built on top (eg. .net, although a wonderful system, becomes more and more incompatible with the underlying OS; eg. GDI+).

Microsoft started open sourcing a lot of their developer frameworks etc (ASP.Net MVC in 2012, Entity Framework in 2013 etc) and we saw fairly large shifts in developer conferences and support

By your own words, we can infer that Ballmer's middle-empire period required large shifting from where it was to what it became.

Ballmer wasn't bad; his jumping around on stage shouting "Developers!" showed that he knew what the true value of Windows was: the external developers who wrote Win32 code for retail products or company-internal developers. However, his middle-empire stage was a shift to focusing on selling to enterprise customers. This isn't a bad things by itself, but by taking his eye off the "Developer!" ball and focusing elsewhere, he guaranteed that plenty of developers went elsewhere. For example, with the death of VB plenty of developers shifted to Java rather than .net. The fact that it needed a large shift in support shoes just how far developers had slipped in Microsoft's priority.

It's interesting to see how Nadella is shifting the focus again and broadening it (Windows 10 on Raspberry Pi, for example). Time will tell if Nadella is simply being an anti-Ballmer or if this glasnost is signs of a more fundamental shift in the way Microsoft does business. I hope it's the latter.

Re:Developers, Developer, Developers

[omtinez]

Python list comprehension, which is OK for simple queries but doesn't scale as nicely as LINQ

Re:Developers, Developer, Developers

[BaronAaron]

Depending on what dependencies you use in your code you could have had it running on Linux years ago using Mono.

I've had good experiences running ASP.NET MVC and console apps on Linux in production environments.

Re:Developers, Developer, Developers

[Anonymous Coward]

VB6 and FoxPro served their purpose and needed to end. They were prototypes for various parts of what became .Net.

VB6 was discontinued right away when Microsoft combined VBRUNxxx.DLL with their Java implementation that got shitcanned by the anti-trust courts. Then they refined it into a common language runtime for a VB variant, a Java variant, and a heavily modified variant of C and all of the object oriented stuff that has cropped up around it. Later, they dropped the Java variant. Later still, they added a functional programming language.

FoxPro was discontinued when they released LINQ, which basically mimics the one thing FoxPro had going for it that nothing else did.

Win32 has not become "irrelevant", since all of the newer technologies still rely on those older ones. (.Net "winforms" simply packages up the old Win32 WNDCLASSEX and window class registration and instantiation into a handy Form object, so instead of 80+ lines of boiler-plate code, you use a simple new Form() and be done with it. I fail to see how this is anything but progress.)

And .Net uses GDI and GDI+ directly, and has done so since day 1. More recently, it also uses Direct3D and Windows Media Framework directly.

It's actually rather amazing that otherwise well-informed people on tech sites keep parroting this panicky crap that naysayers said years ago without doing any research into whether it's correct or not. It seems that the one thing Microsoft hasn't been able to change is the level of trust people place in them. A good portion of that was earned, but it's also about time that some people at least brush a few of the chips off of their shoulders.

Re:Developers, Developer, Developers

[neilo_1701D]

VB6 was discontinued right away when Microsoft combined VBRUNxxx.DLL with their Java implementation that got shitcanned by the anti-trust courts.

I though it was because VB6 was a COM product?

... Later still, they added a functional programming language.

Nice sledge :)

Win32 has not become "irrelevant", since all of the newer technologies still rely on those older ones. (.Net "winforms" simply packages up the old Win32 WNDCLASSEX and window class registration and instantiation into a handy Form object, so instead of 80+ lines of boiler-plate code, you use a simple new Form() and be done with it. I fail to see how this is anything but progress.)

And .Net uses GDI and GDI+ directly, and has done so since day 1.

I completely agree with everything you've written... if you're a .net programmer, you're fine. But if you're a C programmer sitting directly on top of Win32, you're screwed

My point, and I guess I didn't make it clearly, is that although these new technologies are fantastic, there is older code out there that is company-critical to companies who invested heavily in the creation of a solution, based on Microsoft's past history of obsessive backwards compatibility, and now find themselves with no way forwards, other than a costly rewrite.

Re:Developers, Developer, Developers

[Dutch Gun]

I completely agree with everything you've written... if you're a .net programmer, you're fine. But if you're a C programmer sitting directly on top of Win32, you're screwed

Lots of cutting-edge applications are still written in C or C++ and directly use the Windows API (it's not called Win32 anymore). In fact, I'd go so far as to say the majority of large commercial applications you can think of are native apps: Photoshop, Microsoft Office, most videogames, web browsers, media players, etc, etc. Nearly all the new APIs released with new versions of Windows are available to native applications. There's no reason a company has to abandon their legacy C codebase if they don't want to. BTW, the "way forward" for C developers is called C++, and it's conveniently backwards compatible with your C codebase.

The major reason one would consider switching to .NET from native code is productivity, not new functionality. The .NET APIs are much easier to use than the much older Windows APIs, and the languages .NET supports like C# are far easier to use as well. You don't even have to completely abandon your C or C++ code either. It's pretty simple to write interop layers to communicate between C# and C, and have done so many times at work.

Re:Developers, Developer, Developers

[shutdown -p now]

The nice thing about LINQ is that it covers more ground than just basic map/filter/reduce; notably, joins - which makes it possible to actually reasonably translate expression trees to SQL and such. And joins are a pain in the ass to write using raw lambdas, the sugar is very welcome there.

Deep nested queries are also often convenient with sugar, because it has "let" for temporary values.

Anticipating RMS response

[krel]

I anticipate Stallman may have a heart attack soon.

Re:Anticipating RMS response

[cb88]

RMS... doesn't like LLVM either.

http://developers.slashdot.org/story/15/02/08/210241/rms-objects-to-support-for-llvms-debugger-in-gnu-emacss-gudel

I don't understand

[Blaskowicz]

With LLVM using an intermediate representation of code (LLVM IR) and CLR another : MSIL, now called CIL, does that mean it goes C# -> LLVM bytecode -> .NET bytecode?, does the JIT does both steps at once, why doesn't that mean every single language with a LLVM target can now run on the CoreCLR?, was LLVM modified, was what's in my first question horribly wrong?

Re:I don't understand

[Anonymous Coward]

Other direction; the C# -> .NET bytecode step happens elsewhere, this project handles the .NET bytecode -> LLVM bytecode part:

LLILC is an Open-Source project that Compiles msIL (.NET) code to native binary, using the LLVM compiler framework. We pronounce it "lilac". The project will provide both a JIT ("Just-In-Time") and an AOT ("Ahead-Of-Time") compiler targeting CoreCLR.

From the Background page of the linked wiki.

Re:I don't understand

[HyperQuantum]

I didn't look into the details of the project, but to me it seems to like the following:

A JIT compiler is used in a virtual machine to run a function/method by compiling it from some sort of bytecode or intermediate representation to native code and then jumping to the generated code to execute it. So in this case this would be when the Common Language Runtime wants to run a CIL method for the first time: it generates LLVM intermediate representation from the CIL, then uses LLVM to compile that to native code.

So it would be: CIL --> LLVM data --> native code

This means that the CLR, and thus all code compiled for .NET, can run on all platforms that LLVM can generate native code for.

Re:I don't understand

[complete loony]

C# -> MSIL / CIL (CLR) -> LLVM bitcode -> machine code.

The summary is slightly misleading. Though they are working towards using LLVM, they currently have about 90% of their core JIT tests working with LLVM on windows x64, the rest fall back to their current JIT. So you won't be able to use this to run C# on linux / arm for a while yet.

There's been quite a bit of recent development on JIT support in LLVM. They had an old JIT a few versions ago which had it's own machine code generation pipeline. This was replaced with MCJIT, that leveraged the same code generation pipeline as the AOT path. More recently there's an effort called ORC to build a more flexible API to better handle the use cases that various JIT writers typically wish to implement. Their existing MCJIT API will still exist but it will become just one example implementation of using this underlying API.

While it is possible to implement garbage collection and exception handling in LLVM, these areas of the compiler could benefit from more improvement. Particularly in the area of compatibility with the native Microsoft stack.

Make sense, actually

[Dutch Gun]

I think it's important to understand that the .NET JIT compiler should probably be considered more part of the .NET *runtime*, not necessarily part of the development platform for .NET. Since they want to port .NET to non-Microsoft operating systems, it makes sense to utilize LLVM to target those platforms for the JIT compiler rather than trying to write a new one from scratch. They needed a solid compiler to accompany their open source .NET platform for it to be a more complete open-source solution. Moreover, they've been extending Visual C++'s support for alternative platforms like Android, so it also makes sense that they'd be gaining expertise with LLVM.

It's probably not the end of their proprietary compiler, or even necessarily an indication they're thinking this way, but it may make more sense for them to utilize LLVM so as to target a larger number of platforms. They just recently rewrote their own .NET compiler a couple of years ago and released it as open source, so it's sort of odd to see a new project so soon. I'm guessing they figured it would be more work to extend that project to support all the platforms they're releasing CoreCLR for than using LLVM. Hard to say.

Also, there's still the native compiler, used for C/C++, and they've been sinking an enormous amount of development resources into making it compliant with the recent advances in those languages, so it also seems unlikely they're going to toss that work.

This makes a LOT of sense for Microsoft

[jonwil]

Right now Microsoft has a JIT compiler running on a few platforms that translates .NET byte code into native code. Instead of reinventing the wheel and writing their own JIT compiler for a bunch other platforms they want to be able to run .NET code on, they are instead using something that already exists in the form of LLVM.

They aren't abandoning anything, just using LLVM instead of rolling their own JIT compiler on certain platforms where doing so makes sense.