Slashdot Mirror
Microsoft Announces R Tools For Visual Studio (technet.com)
theodp writes: A year after its acquisition of Revolution Analytics, Microsoft announced a slew of R-related product offerings, and noted that Revolution R Open is giving up her maiden name and will henceforth be known as Microsoft R Open. Tucked away in the announcement was the news that R is coming to Visual Studio. Microsoft has released a teaser video for R Tools for Visual Studio (RTVS) and is taking sign-ups for early access.
R is pretty much pure statistics. While it has a built-in interpreter to load data from csv files or user input or whatever and then run its functions against the data set, it really shines as a library to be used in other "real" programming languages where you have logic, loops, etc. available to you.
And since there are R interfaces for Python (http://rpy.sourceforge.net/) it isn't a "versus" situation.... what a bargain!
Don't blame me, I voted for Kodos
R is preferred by statisticians. Many statisticians are on the leading front of creating new traditional machine learning algorithms (not the GPU driven or map reduce stuff "hip" companies are dealing with right now). Things like supervised classification tasks and clustering algorithms. This usually means you have access to a researchers implementation of a new algorithm fairly quickly, long before it's in a commercial package. It also means you have to deal with a lot of 1-off code and deciding whether their function wants a row-vector or column-vector.
Python seems to be much more popular with those having a computer science background. There are far fewer machine learning algorithms available in Python. However, if you are going to design a large system, it's generally much more convenient to do in Python. There are Python interfaces to R as well.
Julia is new on the scene and attempts to solve the shortcomings of Python and R (insert xkcd comic here). Performance is good and has interfaces to many languages. I've used it a few times and it's maturing, but it's definitely risky doing any long term project in Julia.
Then there's Java. Weka is a popular machine learning package with a GUI and all of the algorithms available as jar files. Very consistent API and includes pre-processing tools. Weka also has a marketplace for new algorithms. However, many times you just have to write a 1-time script for data cleanup or to compare algorithms, and it's definitely not convenient to do in Java. I haven't seen many pure Java people doing this type of work in the wild. The final implementation may end up in Java, but the initial work seems to almost always be in R and Python.
But, R is all about stats. It has great charting and analysis libraries, far exceeding those that exist in Python or any other general purpose language.
SAS is kind of the corporate standard if you want long term maintainability and a large selection of potential workforce...but it is expensive and if I were starting out today, I am not sure I would pick it. It is however much more easy to scale to huge datasets than R...SAS pretty much works as long as you can fit the data on your hard drive. No need to fuck with breaking up projects into small pieces or investing in boxes with 1TB of RAM. Millions and billions of observations are totally OK as long as you are willing to wait for the program to finish running.
R has been making inroads at replacing SAS and Stata as the teaching language of choice (mostly because it is free)...so it is probably easiest to find straight college grads with some R experience than anything else.
Bottles.
MATLAB is amazing for general 'data science,' and is very widely used for certain tasks, such as image processing. It provides a huge array of already-implemented algorithms for computer vision, statistics, machine learning, and simulation. Many academic labs use it, and many students receive MATLAB training. On the other hand, MATLAB is proprietary and quite expensive. (It's semi-open source because most of it's functions are MATLAB scripts themselves). The language is very readable, except maybe the native array syntax, and comes with extremely good documentation, but it's clunky for general purpose programming. It has an OK IDE and one of the best debuggers in any language. The runtime is redistributable, so you *can* make portable applications, but again, it's a little clunky. The open-source GNU Octave and Scilab environments are also (mostly) code-compatible with MATLAB. All-in-all, it scores highly in all three aspects you mentioned, but it's very expensive.
Python is also very good, once numpy, scipy, matplotlib, pandas and ipython/jupyter packages are installed. Like MATLAB, Python is widely used in academia, and lots of students receive training. There are many function/algorithms already available, but somewhat less so than in MATLAB. For example, the statistics capabilities are similar, but MATLAB has more image processing functions. Plotting and visualization also haven't quite caught up to MATLAB yet. Python has the great advantage of being totally free and open-source, and there are a large number of IDEs and debuggers available. Python is also a great general purpose language for self-contained, portable applications that may grow out of data analysis code. The documentation can be lacking in some modules, but there's good free support online via e.g. stackoverflow. Python is readable and easy to learn. It scores about the same as MATLAB, weaker in some areas, stronger in others, and is completely free. There's active development of the analytics modules and going forward Python will probably become more popular for data science.
R is a bit of a special case. It has excellent statistics and machine learning capabilities, and there are a lot of extension packages available with specialized features, but it's really not as general as MATLAB or Python. I'm unaware of anyone using R for image processing, for example. As a language, it's very declarative, and the analyst doesn't need to understand statistics methods or their implementations in order to use them. That's great for beginners and convenient for experts, but can lead beginning/intermediate users astray if they don't appreciate the distinctions between significance and effect size, between different measures of significance/effect size, independence of variables, etc. Plots and visualizations in R tend to look nice when printed as PDF, but they're essentially non-interactive. R isn't general purpose at all, and personally I don't like its language conventions. I had the same experience with Mathematica, some people really like it and it's great for certain things, but I just can't stand the language. Back to R, I think the usefulness is great for statistics, less so for other tasks. Maintainability is OK - IMHO the language is not as intuitive as MATLAB or Python. My impression is that fewer people receive training with R, and it's a little less popular in general. It's the only one of these three languages I didn't see until grad school.
My first choice for any new data analysis task is Python. I think it has the brightest future, and it's available to everyone for free. I'll use MATLAB if one of its built-in functions will save me a ton of time, or if I need to prototype something very rapidly (I guess it's still my strongest language). R I only use if I absolutely need something from one of its third-party modules. Lately, I've been experimenting with Julia, but it's not close to mature enough for my academic projects, let alone commercial ones. Sometimes I use external visualization tools, like LLNL VisIt, if I need to make high-quality, interactive visualizations of very large data sets. Hope that helps, sorry for the wall of text.
.: Semper Absurda
I remember back when Cobol was going out of style, and I was an early adopter of C++ (1987-ish). ADA was going to change the world, C++ was doomed to never go anywhere, and C was going to vanish. Yourdon wrote a book about the fall of the American programmer. I wept over my keyboard. I told everyone I was crying because my C++ compiler was so frigging slow. But I knew the world was going to change, that ADA was going to kill all the other languages, and I really loved working in C and C++. So I waited for the world to change. Prolog was a big deal about the same time, and I didn't want to miss out, so I jumped on it for functional. And the "wow" thing of the day was Expert Systems. They were going to change the world. So I wrote some interesting diggers with Prolog. And I waited for the world to change. In around 1992 I entered the CHICAGO beta with Microsoft in preparation for Windows NT (which was going to change the world). I even wrote a device driver for CHICAGO to operate a RHETOREX PCM telephony board, and a printer driver for an old ATARI thermal printer. Fun projects, actually. Didn't make a dime, though. OS/2 WARP came out around then. It was going to change the world. It was 1994 when I first saw Java. It was going to change the world. I looked at the language, and it didn't interest me: I had C++, and C++ was starting to grow. And I couldn't even imagine not having pointers, not being able to talk to the CPU or devices directly (sans imported libraries). 1995 came along, a friend handed me a stack of floppies (I think about 20), and installed SLACKWARE LINUX over my Windows partition. "This is going to change the world," he said. It was funny, but I really and for truly was convinced this time that the world would be changed, and I didn't wait. I jumped into Linux with everything I had, and I've been working in C and C++ in linux ever since. I'm not trying to be funny or anything. The truth of the matter is that I've listed only several languages here, but I've worked in at least two dozen others that probably most people have never heard of (e.g. SPL for MTM/32). I keep seeing language come and go, that are supposed to change the world. As a young engineer I'd jump on every new language that came out, but most of the time the language turned out to be raspy in some way, was good at exactly one thing, and pretty sucky at everything else. And here we are. 2015. I still work in C and C++ every day of the work week, but I don't see ADA anywhere, I haven't cranked a line of FORTRAN since 1993, I never had to write RPG for a living, I've avoided Cobol altogether, HASKELL never took off like it was supposed to (ditto EIFEL), MATLAB costs too much (even though it is a heck of a tool), I like Python and don't much care for Perl, and on and on and on. And I've debugged way more Java code than I ever wanted to, but I haven't written a single line of Java, yet. And here's what I wanted to get to... I opened up Slashdot today and found the OP's article, and watched the video. And you know what? THIS ISN"T ANYTHING NEW! Not the features, not the tools, not the results. It is yet another language, yet another IDE, and I'm seeing the same kind of features I was using back in the 90's. Funny thing... I use gcc/g++ for my compilers; I use VIM for my editor; and I do quite well. I hate IDE's with a passion; and any time I've been sentenced to use a product with "code completion" or "intellisense", I feel like I've joined some kind of Commune of the Damned. I've quit jobs to escape the transition so the baloney world of IDE productivity. Maybe that means I'm out of touch or old fashioned or "stuck in the 80's". But I've never wanted for a job. And the kids we interview today mostly know the current "hip" language(s) and/or IDE (Hey! lets write a web page, yah!), but if you ask them about superscalar architectures, or how to write a Fibonacci generator using C++ templates, or what a 3-way handshake is for, you get a deer-in-the-headlilghts stare.
I've got no problem with that. These companies are in business to make money. They aren't charities, and I wouldn't expect them to give away the software that provides most of their income. If they did, they'd go out of business.
But consider what you said. Just because it costs them nothing to open source something, and there's no benefit to keeping it closed, that still isn't a reason to open source it. They could just as easily keep it closed anyway. And in the old Microsoft, which saw open source as evil and did everything they could to discredit it, that's exactly what they would have done. Their new attitude seems to be, "If it doesn't hurt us to open source something, then sure, let's go ahead and do it." That's a big change.
"I'm too busy to research this and form an educated opinion, but I do have time to tell everyone my uninformed opinion."
Well, it's because they have a new boss on top. The first boss said sharing was bad and you're an evil person if you did (Bill's famous letter to hobbyists in the 70s). The second one basically kept "Microsoft is dominant and we will rule" while the current one is more humble and akin to "OK, we have PCs, but PCs are not the be-all-end-all computing device anymore, and while we make a lot of money here, it's a mature product and it won't last". It's why they're putting Office everywhere - even on non-Windows things - you can have it on your smartphone, your tablet, etc. And while it's not necessarily great for producing content, if you need to update a slide on the way to the customer, well, that's a sale of Office365 subscriptions, because they can do it in under a minute, rather than try to dig out their laptop, start it up, etc.
If anything, the new Microsoft realizes its no longer the central part of everyone's lives - you don't need a PC for everyone in the household (and likewise a Windows and Office license) - a lot, if not most, are perfectly happy to just use a smartphone and tablet, and maybe from time to time, use the shared PC.
So it's busy finding new ways to be a part, to make itself relevant again. Not a bad thing, really - if you were wedded to a Microsoft solution, it just means Microsoft is now offering more ways to stay Microsoft only. If you're not, well, maybe it will attract you into the fold.