Slashdot Mirror
Microsoft Announces R Tools For Visual Studio (technet.com)
theodp writes: A year after its acquisition of Revolution Analytics, Microsoft announced a slew of R-related product offerings, and noted that Revolution R Open is giving up her maiden name and will henceforth be known as Microsoft R Open. Tucked away in the announcement was the news that R is coming to Visual Studio. Microsoft has released a teaser video for R Tools for Visual Studio (RTVS) and is taking sign-ups for early access.
Damn, I am beginning to think that MS might actually be starting to take this whole free software thing seriously.
Many Visual Studio shops are also Microsoft only shops where open source is frowned upon and use of tools not included in the standard install is verboten. Microsoft is doing this to try and get tools into the hands of people who otherwise wouldn't be able to access them on closed corporate networks. However, even this isn't always effective since some managers are in the nasty habit of banning certain languages, features or other parts of an otherwise "standard" install that they don't like, probably because they're worried that some new hire will write some mission critical program in "R" and that when he's gone in a year or two they will have to pay consultants huge amounts of money to maintain or re-write the app in a language that's easier to hire for.
Embrace, Extend Extinguish
'nuf said.
I read the Embrace, Extend and Extinguish article on Wikipedia looking for examples of this actually happening, and it had to go back 15 years to find an example. I think you need to find new things to complain about.
-- All that is necessary for the triumph of evil is that good men do nothing. -- Edmund Burke
R is pretty much pure statistics. While it has a built-in interpreter to load data from csv files or user input or whatever and then run its functions against the data set, it really shines as a library to be used in other "real" programming languages where you have logic, loops, etc. available to you.
And since there are R interfaces for Python (http://rpy.sourceforge.net/) it isn't a "versus" situation.... what a bargain!
Don't blame me, I voted for Kodos
R is preferred by statisticians. Many statisticians are on the leading front of creating new traditional machine learning algorithms (not the GPU driven or map reduce stuff "hip" companies are dealing with right now). Things like supervised classification tasks and clustering algorithms. This usually means you have access to a researchers implementation of a new algorithm fairly quickly, long before it's in a commercial package. It also means you have to deal with a lot of 1-off code and deciding whether their function wants a row-vector or column-vector.
Python seems to be much more popular with those having a computer science background. There are far fewer machine learning algorithms available in Python. However, if you are going to design a large system, it's generally much more convenient to do in Python. There are Python interfaces to R as well.
Julia is new on the scene and attempts to solve the shortcomings of Python and R (insert xkcd comic here). Performance is good and has interfaces to many languages. I've used it a few times and it's maturing, but it's definitely risky doing any long term project in Julia.
Then there's Java. Weka is a popular machine learning package with a GUI and all of the algorithms available as jar files. Very consistent API and includes pre-processing tools. Weka also has a marketplace for new algorithms. However, many times you just have to write a 1-time script for data cleanup or to compare algorithms, and it's definitely not convenient to do in Java. I haven't seen many pure Java people doing this type of work in the wild. The final implementation may end up in Java, but the initial work seems to almost always be in R and Python.
Microsoft actually has no other product like R, so I would imagine the real reason they bought R was so they could add an additional layer to their data analysis tools inside SQL server. It's actually a no-brainer. Do a SQL server query, store results in a data frame, run a clustering algorithm against the data to see purchasing habits.
Billions of dollars, and they can't think-up a name that sounds a bit less stupid?
'S' (for "Statistics" - originally with single quotes, those are usually being dropped now) is a programming language created in 1975-1976 at Bell Labs (which had a tradition of single letter named programming languages, such as C) on General Electrics GCOS mainframes and since 1979 on UNIX.
R is an implementation of 'S' (with lexical scoping semantics inspired by Scheme) since 1993.
/. refugees on Usenet: news:comp.misc
If you know both a Scripting language eg. Perl or even bash, and a spreadsheeting program, then R is almost unbelievably easy to learn, I was doing semi "advanced" stuff in less than a day, and I am not that bright.
The real problem will be understanding the specific statistics you are wilding, which can be a lot harder.
this is one of the nicer intro's I have found but tastes vary
http://www.statmethods.net/
Not 'nuf. You missed a 'omma and an 'eriod.
Confucius say, "Find worm in apple - bad. Find half a worm - worse."
But, R is all about stats. It has great charting and analysis libraries, far exceeding those that exist in Python or any other general purpose language.
SAS is kind of the corporate standard if you want long term maintainability and a large selection of potential workforce...but it is expensive and if I were starting out today, I am not sure I would pick it. It is however much more easy to scale to huge datasets than R...SAS pretty much works as long as you can fit the data on your hard drive. No need to fuck with breaking up projects into small pieces or investing in boxes with 1TB of RAM. Millions and billions of observations are totally OK as long as you are willing to wait for the program to finish running.
R has been making inroads at replacing SAS and Stata as the teaching language of choice (mostly because it is free)...so it is probably easiest to find straight college grads with some R experience than anything else.
Bottles.
MATLAB is amazing for general 'data science,' and is very widely used for certain tasks, such as image processing. It provides a huge array of already-implemented algorithms for computer vision, statistics, machine learning, and simulation. Many academic labs use it, and many students receive MATLAB training. On the other hand, MATLAB is proprietary and quite expensive. (It's semi-open source because most of it's functions are MATLAB scripts themselves). The language is very readable, except maybe the native array syntax, and comes with extremely good documentation, but it's clunky for general purpose programming. It has an OK IDE and one of the best debuggers in any language. The runtime is redistributable, so you *can* make portable applications, but again, it's a little clunky. The open-source GNU Octave and Scilab environments are also (mostly) code-compatible with MATLAB. All-in-all, it scores highly in all three aspects you mentioned, but it's very expensive.
Python is also very good, once numpy, scipy, matplotlib, pandas and ipython/jupyter packages are installed. Like MATLAB, Python is widely used in academia, and lots of students receive training. There are many function/algorithms already available, but somewhat less so than in MATLAB. For example, the statistics capabilities are similar, but MATLAB has more image processing functions. Plotting and visualization also haven't quite caught up to MATLAB yet. Python has the great advantage of being totally free and open-source, and there are a large number of IDEs and debuggers available. Python is also a great general purpose language for self-contained, portable applications that may grow out of data analysis code. The documentation can be lacking in some modules, but there's good free support online via e.g. stackoverflow. Python is readable and easy to learn. It scores about the same as MATLAB, weaker in some areas, stronger in others, and is completely free. There's active development of the analytics modules and going forward Python will probably become more popular for data science.
R is a bit of a special case. It has excellent statistics and machine learning capabilities, and there are a lot of extension packages available with specialized features, but it's really not as general as MATLAB or Python. I'm unaware of anyone using R for image processing, for example. As a language, it's very declarative, and the analyst doesn't need to understand statistics methods or their implementations in order to use them. That's great for beginners and convenient for experts, but can lead beginning/intermediate users astray if they don't appreciate the distinctions between significance and effect size, between different measures of significance/effect size, independence of variables, etc. Plots and visualizations in R tend to look nice when printed as PDF, but they're essentially non-interactive. R isn't general purpose at all, and personally I don't like its language conventions. I had the same experience with Mathematica, some people really like it and it's great for certain things, but I just can't stand the language. Back to R, I think the usefulness is great for statistics, less so for other tasks. Maintainability is OK - IMHO the language is not as intuitive as MATLAB or Python. My impression is that fewer people receive training with R, and it's a little less popular in general. It's the only one of these three languages I didn't see until grad school.
My first choice for any new data analysis task is Python. I think it has the brightest future, and it's available to everyone for free. I'll use MATLAB if one of its built-in functions will save me a ton of time, or if I need to prototype something very rapidly (I guess it's still my strongest language). R I only use if I absolutely need something from one of its third-party modules. Lately, I've been experimenting with Julia, but it's not close to mature enough for my academic projects, let alone commercial ones. Sometimes I use external visualization tools, like LLNL VisIt, if I need to make high-quality, interactive visualizations of very large data sets. Hope that helps, sorry for the wall of text.
.: Semper Absurda
Exactly, nobody really care that much about skills with a language when the set of skills you are really looking at is statistics. The R user is doing statistics while most of the python users don't know a fuck about statistics. So, if you are seeking for a stats guy and are too stupid to ask for it namely, you may ask for a R guy and get someone who knows stats. With python, you will get someone knowing python.
Achille Talon
Hop!
I remember back when Cobol was going out of style, and I was an early adopter of C++ (1987-ish). ADA was going to change the world, C++ was doomed to never go anywhere, and C was going to vanish. Yourdon wrote a book about the fall of the American programmer. I wept over my keyboard. I told everyone I was crying because my C++ compiler was so frigging slow. But I knew the world was going to change, that ADA was going to kill all the other languages, and I really loved working in C and C++. So I waited for the world to change. Prolog was a big deal about the same time, and I didn't want to miss out, so I jumped on it for functional. And the "wow" thing of the day was Expert Systems. They were going to change the world. So I wrote some interesting diggers with Prolog. And I waited for the world to change. In around 1992 I entered the CHICAGO beta with Microsoft in preparation for Windows NT (which was going to change the world). I even wrote a device driver for CHICAGO to operate a RHETOREX PCM telephony board, and a printer driver for an old ATARI thermal printer. Fun projects, actually. Didn't make a dime, though. OS/2 WARP came out around then. It was going to change the world. It was 1994 when I first saw Java. It was going to change the world. I looked at the language, and it didn't interest me: I had C++, and C++ was starting to grow. And I couldn't even imagine not having pointers, not being able to talk to the CPU or devices directly (sans imported libraries). 1995 came along, a friend handed me a stack of floppies (I think about 20), and installed SLACKWARE LINUX over my Windows partition. "This is going to change the world," he said. It was funny, but I really and for truly was convinced this time that the world would be changed, and I didn't wait. I jumped into Linux with everything I had, and I've been working in C and C++ in linux ever since. I'm not trying to be funny or anything. The truth of the matter is that I've listed only several languages here, but I've worked in at least two dozen others that probably most people have never heard of (e.g. SPL for MTM/32). I keep seeing language come and go, that are supposed to change the world. As a young engineer I'd jump on every new language that came out, but most of the time the language turned out to be raspy in some way, was good at exactly one thing, and pretty sucky at everything else. And here we are. 2015. I still work in C and C++ every day of the work week, but I don't see ADA anywhere, I haven't cranked a line of FORTRAN since 1993, I never had to write RPG for a living, I've avoided Cobol altogether, HASKELL never took off like it was supposed to (ditto EIFEL), MATLAB costs too much (even though it is a heck of a tool), I like Python and don't much care for Perl, and on and on and on. And I've debugged way more Java code than I ever wanted to, but I haven't written a single line of Java, yet. And here's what I wanted to get to... I opened up Slashdot today and found the OP's article, and watched the video. And you know what? THIS ISN"T ANYTHING NEW! Not the features, not the tools, not the results. It is yet another language, yet another IDE, and I'm seeing the same kind of features I was using back in the 90's. Funny thing... I use gcc/g++ for my compilers; I use VIM for my editor; and I do quite well. I hate IDE's with a passion; and any time I've been sentenced to use a product with "code completion" or "intellisense", I feel like I've joined some kind of Commune of the Damned. I've quit jobs to escape the transition so the baloney world of IDE productivity. Maybe that means I'm out of touch or old fashioned or "stuck in the 80's". But I've never wanted for a job. And the kids we interview today mostly know the current "hip" language(s) and/or IDE (Hey! lets write a web page, yah!), but if you ask them about superscalar architectures, or how to write a Fibonacci generator using C++ templates, or what a 3-way handshake is for, you get a deer-in-the-headlilghts stare.
Same amount of time it took Visual Studio to kill gcc.
In my opinion, if you're coming from a generic software development background, you will really prefer Python over R for pretty much anything that GP has listed. R is a very... quirky language, with an even quirkier standard library, and will quickly drive someone used to the familiarity and comfort of most mainstream languages nuts.
Full disclosure: I am a developer on both PTVS and RTVS (different but intersecting subsets of the same team work on both).
So why has no one mentioned RStudio yet? We just seem to be talking R. This is pretty much a clone of RStudio so far, with *slightly* better code-completion.
And a license that is not Affero GPL v3.
I think you will find that there are other differences that you may find interesting and useful beyond that, though. The video is a very brief overview, and doesn't show everything in detail, but some things are already visible. For example, notice how at 1:15, the history brings up the entire multiline expression, that is editable as such (rather than scrolling through each individual line of it).
MS tools for open languages rarely give anything I can't get elsewhere, just the same stuff over their own tooling. I remember them pitching Python tools as if they invented the first IDE with code-completion for Python while I had been using tools with equivalent functionality for 10 years prior.
With respect to "first IDE with code completion" - I'm not aware of it having ever been pitched like that.
What it had been pitched like - and what was true - is that it's the IDE with the best code completion, especially on large volume of pure Python code (as opposed to C modules where all the completion data comes from doc comments anyway). Until it and PyCharm showed up, most Python IDEs were doing very basic completion based on function names and such.
The big difference we made was adding an analysis engine that actually keeps track of the flow of values throughout the source code, across module boundaries even; and tracking more than just classes, but also e.g. individual values of tuples and dicts. This enables good completion for many common dynamic patterns that conventional identifier-matching approach cannot handle. And because the flow goes both ways, you can, for example, call a function in one place of your program, and then when you're editing the function, you will see the types of arguments, and the corresponding members, based on all its call sites.
Have a look at this, and tell me which IDE that you've used 10 years ago could do it back then. Heck, most Python IDEs out there can't do it today! Code completion for a dynamic language is hard.
As far as "give anything I can't get elsewhere" - how about this?
"That's not entirely the way I remember it. Microsoft didn't touch ODF, didn't extend or extinguish it. They tried to bypass the need for it."
You don't think Microsoft people are stupid, nor their marketing guys haven't thought on what made Microsoft a successful company and what "embrace, extend, extinguish" really means and how it works, right?
Embrace: Oh! you people really want "open standards"? Like ODF? I'll give you one.
Extend: Here you have an open standard just like you want them: docx. And I'll make sure I get docx to be approved and to be in a position of being chosen by EU governments and whoever says that needs and open standard. Certainly I'll add little things to these kind of standards, like binary blobs or "this is to be implemented just like in Word 6" paragraphs, so it becomes impossible to re-implement.
Extinguish: Oh, by the way, in the process of making docx an open standard, I'll corrupt the standard-approval process so much that no one ever will ask for anything-open again.
In addition, they 'embraced' ODF itself by providing an ODF format option in MSOffice that didn't work with any other ODF implementation - by intentionally 'interpreting' vague aspects of the spec as 'anything but what's already been implenented in OpenOffice'. At the same time, releasing an MSOffice implementation of OOXML that differs from the published standard.
Not exactly playing nice. More like polluting an existing standard while waiting for their monopoly magic to entrench their new pseudo-standard. Kind of like Java vs C#. And for the C# fans out there, it's not necessarily that their stuff isn't ever any good (though if excess complexity is a negative, OOXML's gotta be one of the worst standards ever). The point is that Microsoft could well embrace true open standards - even while keeping its implementations closed source, and even possibly coming out on top with the best implementation. But time and again, they try to subvert open standards. The simple fact that they're losing some of their ability to do it these days doesn't change a thing IMHO.
Posted from my Android phone. Oh, I can change this? There, that's better...