Slashdot Mirror
Deep Learning May Need a New Programming Language That's More Flexible Than Python, Facebook's Chief AI Scientist Says (venturebeat.com)
Deep learning may need a new programming language that's more flexible and easier to work with than Python, Facebook AI Research director Yann LeCun said today. From an interview: It's not yet clear if such a language is necessary, but the possibility runs against very entrenched desires from researchers and engineers, he said. LeCun has worked with neural networks since the 1980s. "There are several projects at Google, Facebook, and other places to kind of design such a compiled language that can be efficient for deep learning, but it's not clear at all that the community will follow, because people just want to use Python," LeCun said in a phone call with VentureBeat. "The question now is, is that a valid approach?" Further reading: Facebook joins Amazon and Google in AI chip race.
Julia has a tremendous problem. It's not designed for users, it's designed for Julia designers. If they had said "let's create an environment for deep learning that is great for threading" everything would be fine. But instead they went with "Hey, let's do all those awesome and cool things that we always wanted to see in a programming language and also it should be great with deep learning and implicit threading". The result is a (possibly great) environment that takes far too long to learn, has way to many individual quirks and ways of doing things that differ from the standard approach, and just is a bitch to intuitively understand.
The environment may or may not be great, but the designers made sure that you couldn't just pick it up and go, you have to go "ahaaaa so that's how you do that" for every single thing. And when the option is using Python/R that you already understand or use Julia that you have to learn from scratch, the choice is easy, especially for the people that are scientists and not programmers at heart - which is the exact audience that Julia is targetting.
Just use C++
Indeed, and the library you want in particular is called DLib.
http://dlib.net/
Specifically:
http://blog.dlib.net/2016/06/a...
the networks are represented as templates. It's pretty cool and very high performance. Particularly impressive given the relative resources invested relative to Tensorflow and PyTorch/Caffe.
SJW n. One who posts facts.
Futhark looks cool: https://futhark-lang.org/ and promising in this realm.
"Futhark is a small programming language designed to be compiled to efficient parallel code. It is a statically typed, data-parallel, and purely functional array language in the ML family, and comes with a heavily optimising ahead-of-time compiler that presently generates GPU code via CUDA and OpenCL"
the ML family of languages being things like Standard-ML, Haskell, OcaML.
You can write code similar to Matlab
MATLAB is a terrible example of an "easy to learn" language. It's full of shitty hacks like "putting a semicolon after an expression suppresses output; deleting the semicolon causes the expression to dump its output to console." It's loaded with arcane semantics like the differences between a normal array and a cell array. For fuck's sake, it permits semicolons in the middle of a parameter list, like this.
MATLAB has the quintessential property of an overdesigned language, which is: if I leave it alone for a few months and come back to it, I have to re-learn the whole damn thing. The syntax, the library, and the UI are all unintuitive and illogical. I rely heavily on my cheatfile that I've built up from this iterative exercise to get back up to speed faster. I don't have to do that with Python or C.
or Numpy
Numpy is a terrible example of an "easy to learn" API. Numpy arrays use semantics that are logically similar to Python arrays - but of course they're nothing like each other, none of the libraries work the same way, etc. And And some core features are so poorly documented that you have to dig through sample code until you find something that vaguely resembles what you wanna do, and then shoehorn it into the shape you want. And if you want to stare into the abyss of insanity, try looking into using the Numpy array-sharing options to circumvent the cross-thread limitations of the Python global interpreter lock.
Don't get me wrong: both are powerful and fun when you're acclimated to them. My first Numpy experience was porting a super-simple image processing technique, and my amateur, first-attempt Numpy code finished 10,000 times faster, and that's not an exaggeration. But they're both crappy languages from a model-of-clarity-and-consistency perspective.
I mean, sure, a single data processing pipeline might have to use 6 different conda environments, each with different dependencies and python versions due to tool and libraries are often deprecated with even minor point changes to python versions...... oh yeah, all that and then you have to shoe-horn in tensorflow (or something else).