Slashdot Mirror
Deep Learning May Need a New Programming Language That's More Flexible Than Python, Facebook's Chief AI Scientist Says (venturebeat.com)
Deep learning may need a new programming language that's more flexible and easier to work with than Python, Facebook AI Research director Yann LeCun said today. From an interview: It's not yet clear if such a language is necessary, but the possibility runs against very entrenched desires from researchers and engineers, he said. LeCun has worked with neural networks since the 1980s. "There are several projects at Google, Facebook, and other places to kind of design such a compiled language that can be efficient for deep learning, but it's not clear at all that the community will follow, because people just want to use Python," LeCun said in a phone call with VentureBeat. "The question now is, is that a valid approach?" Further reading: Facebook joins Amazon and Google in AI chip race.
What we need is less Facebook, less Google, less Amazon.
Just use C++
...necessary. All the best thinkers and programmers avoid python. It is imprecise, ineffective, slow as molasses, and is missing many important features. I have seen brave souls try to use it and end up completely abandoning projects for a different language or even give up entirely.
or perhaps prolog...
*** Suerte a todos y Feliz dia!
could no doubt already write better dialogue than any Marvel franchise. Suck it Kendall.
Just use Lisp
Who would have thought such a thing. Haha guess all the CS kids are going to actually have to learn to use languages useful useful in the real world noq
Original post is as informative as a "bad syntax" error message. How is Python inadequate? Is a whole new language really needed or could Python be extended?
Running "against very entrenched desires from against very entrenched desires from researchers and engineers" is not likely to end well for at least a generation*, unless it offers truly spectacular benefits.
*untill said "researchers and engineers" are replaced by history
$CAPTCHA=='lambda" ; lisp rules!
I have one character for you ")"
Unfortunately Procedural and Object Oriented languages, have gotten a foot hold, and training someone to code functionally is quite difficult, and for parallel processing it may be even trickier.
If something is so important that you feel the need to post it on the internet... It probably isn't that important.
or ruby..
"Whenever somebody's conclusion is that we need a new programming language, I know we are dealing with an idiot or more of a philosopher than a programmer."
If indeed f "deep learning" has hit the wall with current languages.... perhaps the hardware itself in it's current configuration isn't up to the task.
Since all languages simply compile to machine code, which runs on an OS making hardware calls, a top level restriction implies a deep level constraint.
To develop the next -real- AI, requires a different hardware approach running some as of yet unconvinced OS - followed by a language.
Go back to Lua! You should never have abandoned it!
ADA think you may be on to something!
... for building Skynet, and it'll be Lisp or perl.
And we all know which one the Lord used: https://xkcd.com/224/
Hire a Linux system administrator, systems engineer,
Nuf said.
No it's not, you just tell them it's like putting a function call to a dll with a return going into an array to into an array location that you check the status of later. Non zero status in the array means the result has occurred. Easy Peasy - loop until your array has the data you need.
Over the past few months I have spoken to a few recruiters I know who were asking me to give them all the senior C++ engineers I know or if I personally was interested.
In what?
Doing complete rewrites of giant mountains of garbage Python code written by twenty something year old hipsters or older researcher type people.
It is boring as fuck work but companies and organizations are desperate and willing to pay huge amounts of money to rid themselves of the clusterfuck that is Python.
I think it fits the requirements quite well.
That's because he isn't sure yet.
Just because you are unhappy with their current tools and know through decades of experience that "more of the same" is not going to improve things, that doesn't mean you know what the future should look like.
sub f{($f)=@_;print"$f(q{$f});";}f(q{sub f{($f)=@_;print"$f(q{$f});";}f});
Like everyone that doesn't truly understand make, they end up creating a retarded subset replacement (xcodebuild, cmake, configure, autoconf, etc.). Besides the GIL problem (which isn't a language but an implementation issue), Python is more than capable of being extended to exceed at any task. It's Touring complete, and already has thousands of useful libraries and a huge user base. Torch, Tensorflow, Caffe, Caffe2, and a dozen other ML libraries aren't enough? AutoML and Karas meta-ML libraries already exist.
it's called Common Lisp.
I am picturing the current crop of 'the past was stupid, never learn from it!' programmers diving into LISP, Smalltalk, or Prolog.
But as always, designing new languages is more fun than learning old ones, and learning from past mistakes makes you uncool.
That, you know, actually edit.
google/jax - this project looks pretty promising
Facecrud Announcement:
New ML language...
...even if specific Solutions are unknown, examples of the problem(s) would be enlightening. I admit that I had not read the linked article when I made my comment. In the context of your polite reply, I did look it over.... and, again I request specific examples of the problem. I do not doubt that problems exist, but would like more information.
Just sayin'.
Everyone is apparently using an interpreted language? Sounds to me like they need to apply the second word of A.I. , since they keep insisting on calling it that.
Its designed to be faster than Python, inherently parallel and aimed at scientists and mathematicians.
See https://www.infoworld.com/article/3241107/python/julia-vs-python-julia-language-rises-for-data-science.html
LISP, then?
I've abandoned my search for truth; now I'm just looking for some useful delusions.
require 'left-pad' console.log('lol')
Or else you'll summon the hoard of JavaScript devs to make your shit pile even steamier and nuttier than before.
I've seen dozens of language and I think Python is the best middle ground.
It would be very, very hard to design something substantially better and easy to learn and versatile.
and for parallel processing it may be even trickier.
WTF are you on about? Functional is the easiest code for parallel processing. Why do you think it has been making such a come back?
Don't use languages designed by language lawyers with esoteric constructs designed to lower productivity of average developers.
A new language is not needed, it's a call to ....
- fund a large project to design a language
- get lots of hype, sell lots of newsprint, magazine space and web blog advertisements,
- give lots of experts speaking fees
- sell lots of training courses online and in person
- sell books
- sell tool sets
- let people attend experts conferences
- help increase the GDP
- provide jobs for PhD holders to research it
-
A language is written for the reader and not the elegance i wet my diaper cool shiny needs of the original developer.
Programming 101 from the 1960s:
"How hard is it extract the business intent of a program by a developer unfamiliar with the system, business processes and computer architecture?"
Programming is not creative writing elegance is not the top goal in computer code, program structure or language.
Another to file in the it's lisp, lisp is good, lisp is the best VS we need to read the program and not be an uber-expert in it.
Story at 11.
Functional is the easiest code for parallel processing.
Nah, C# with they yield style of automatically doing multi-threading is the easiest.
Why do you think it has been making such a come back?
It hasn't. Some of the nice functionality has been imported into Java, but hardly anyone is talking about immutability.
"First they came for the slanderers and i said nothing."
It isn't the programming language, it's the farce that is 'AI'. Good grief, these people are stupid. How Facebook remains profitable is an absolute mystery.
That shit just makes me laugh.
That's definitely what we need. A new programming language.
We suffer more in our imagination than in reality. - Seneca
And WTF are you on about? Your comment makes it seem like you have never done any serious work in a functional language or any serious parallelism.
I don't know, because I'm the exact opposite of Yann LeCun: he's an AI guy unsatisfied with programming languages, and I'm a programming language guy unsatisfied with AI.
I haven't worked in deep learning, so I don't know if he has well-formed complaints or he's just generally unsatisfied.
sub f{($f)=@_;print"$f(q{$f});";}f(q{sub f{($f)=@_;print"$f(q{$f});";}f});
I mean, sure, a single data processing pipeline might have to use 6 different conda environments, each with different dependencies and python versions due to tool and libraries are often deprecated with even minor point changes to python versions...... oh yeah, all that and then you have to shoe-horn in tensorflow (or something else).
you have never done any serious work in a functional language or any serious parallelism.
I've done both.
"First they came for the slanderers and i said nothing."
We heard these exact same people complain about other languages in the 1980s. They complained about Lisp, then Scheme, then Haskell. No one is forcing them to use them, and no one is forcing them to use Python. A General Purpose AI is a pipe dream that will not ever come to fruition, no matter what language you are using. Deep Learning is stuck again because there is no fundamental theory of learning. We have no idea how we learn. Neuroscience will probably discover that learning is a very biological trait, not one that can be copied or simulated in discrete mechanical systems.
If someone develops a language that offers true leverage over Python then the transition *will* happen and all that will be left unanswered is "how long will it take for the transition to happen?"
Guns don't kill people; Physics kills people! - John Lithgow as Dick Solomon on Third Rock From The Sun
I say this every other day in one form or another, but let me try again: we all keep jumping up and down and shouting about whether we need a more flexible language or a fussier language with super-strong type-checking and encapsulation and what not, and some are deeply convinced that Pythons's fascist attitude toward formatting is excellent, and some are not -- one of the reaons the Go project was started, from what I understand, was to get away from syntactic whitespace--
None of us really know who's right about any of this, because to really find out someone would need to do experiments, and designing these experiments would require knowledge of social science techniques, and "Computer Scientists" refuse to consider doing that kind of science, because most of them are really mathematicians in disguise.
So we're stuck with a religious fervour for concepts like "elegance" without any real reason to believe any of it is right.
Norvig doesn't laugh.
Ezekiel 23:20
Do you speak it?
http://programming-motherfucker.com/
Because of Node and it's webfag fanbois proggies became scared of theads because, ya kno...single thread non-blocking my rectum. Now the problems that threads solved never actually went away. Oh noes!!! We need a new language. Perhaps you could use a new gender while you're at it. Fuck Node. Fuck Ruby. Fuck it all. Fuck-ity fuck fuck.
I have one character for you ")"
That's not a character, that's a SIMPLE-STRING of length 1. ;)
Ezekiel 23:20
Nah, C# with they yield style of automatically doing multi-threading
I know next to nothing about C#, but I was under the impression that this was a concurrency mechanism - basically a control flow construct - not a parallelism feature. Did something change?
Ezekiel 23:20
I agree with Yann that our current crop of languages aren't well suited to deep learning, but I'm not sure it's a Python specific problem. I don't think whitespace, threading, syntax etc are the barrier.
It's much more that Python is fundamentally an imperative language, and deep learning doesn't fit into either the imperative or functional category, I really think DL deserves its own category, designed from the ground up for manipulating tensor data structures of unknown shapes.
I haven't come across any language that encapsulate strong typing for tensor shapes, so you end up tracking shapes with pen and paper while debugging your code (and not everything is caught at compile time). Named tensors would be a good start (more information here http://nlp.seas.harvard.edu/Na...) but that wouldn't solve everything. People can't agree on computation graph vs a forward/back functional approach - PyTorch prefers the latter, TensorFlow originally preferred the former (but now seems to be migrating to the latter).
Julia does seem to be the most promising, but that may just be "the grass is always greener" speaking. I haven't played with it myself.
Python has great appeal to academics but it's a very poor choice for any hardcore engineering. It's just inherently slow and, although very readable in small snippets and as glue-code, it's not the best for more complex solutions.
I am not advocating for a whole new language but really think the best bang for the buck would be in simply to better develop the various ML related tools for JavaScript. JavaScript is today clearly what C was back in the 1980's, as Python is what Pascal was. Or like Verilog (industry) verses VHDL (academia). JavaScript (such as V8-based Node.js) compiles to very fast and memory efficient code. Conventional Object-Orientation is inherently slow and memory consuming as it requires an instance factory added to every program and massive memory allocations for each instantiation (using more memory and lots of CPU usage for the OS kernel's memory defragmentation).. JavaScript corrects these errors with its model for Object-Orientation and now also provides a syntax largely identical to that of conventional Object-orientation.
We need to clean up Tensorflow.js and build out data science tools for JavaScript, such as paging data editor to work on ultra-large files piece by piece without having to load them completely into memory.
JavaScript is very well suited for Machine Learning. And it may not hide the underlying machine as do languages like Pascal, Java, and Python (they academics seem to prefer) but there is so much benefit in being true to computation, as it really is.. That is a long discussion on its own.
I do, however, strongly advocate one very important enhancement to JavaScript to greatly simplify asynchrony and to make possible a method of "bug free" programming. Enable the setting of hooks on variable values such that, when modified, will execution a logical condition expression and, if true, execute a code block with a given variable scope... One little bit hook will not slow down execution. For example: // Establishes that, any time x, y, or z values are changed, the expression is evaluation and, // if true, the provided function is executed and the data object "scope" is passed to it. ... }; // kills the "when" hook
var scope = {
var condition = when( x { doSomething() } );
condition = undefined;
This could GREATLY reduce the complexity of things like business logic based on multiple async data queries. It could also make possible, "bug free" programming when using purely for stateful programming between "when" conditions. That is, only syntax errors, system errors, and errors in requirements could go wrong, if all your programming is conditions on states changing states. Overall, software would be vastly more robust than the tendencies today.
Matthew
... there was a programming language designed specifically for that
I'm not convinced that there is a meaningful difference between concurrency and parallelism.
"First they came for the slanderers and i said nothing."
Lisp is used for AI because it supports the implementation of software that computes with symbols very well
More flexible than python? Have these people not seen snakes before? Nothing could be better at tying itself in knots, getting caught in loops, or accidentally eating its own tail ;D
as per subject line
Yep. The runtime starts main(), and returns the return value of main to the OS(). So basically it does START and END.
Everything in between is the responsibility of main(). It needs not interact with the OS at all, other than being started by the OS and telling the OS when it's done. Only for standalone programs, though - kernel modules don't need to do those two things.
There is a very useful kernel module which does nothing but allocate some memory. That's all it does. :)
It's used when you have a a few bytes of dodgy memory. You have that module allocate those memory addresses for it's use - and then do nothing else.
To quote Albert van der Horst (Feb 17 '14 at 13:23) "I would rephrase the question as "what features of a language are necessary for hard AI?" This is a valid and useful question, and the outcome will be that only Lisp, Forth & assembler have what it takes to do hard AI. – "
Which functional languages?
...seriously?
Ezekiel 23:20
Not Invented Here syndrome at work?
People want to keep using Python, it must be doing something right.
On a long enough timeline, the survival rate for everyone drops to zero.
Use ParaSail: https://adacore.github.io/ParaSail/
I seriously question that claim if you think there's no "meaningful difference between concurrency and paralleism," since that statement pretty much implies you have no idea what either of those words mean in the context of program development.
You never know, right?
> There is definitely an expectation of some basic facilities available to your code.
What do you have in mind when you say "some basic facilities"?
Yield in C# is thread safe thus you could use multi-threading/ to iterate through your data set and consume in parallel.
Yeah. Different people have different definitions and from the point of view of the programmer it doesn't really matter if they are happening at the same time or not.
"First they came for the slanderers and i said nothing."
Lisp, Erlang, Haskell. Most with Erlang, least with Haskell. Some people call Python a functional programming language, but it's not really.
"First they came for the slanderers and i said nothing."
LeCun on this story:
For the record, I didn't say "more flexible than Python", I said "More efficient".
https://twitter.com/ylecun
C++ as the replacement for python? The language does not matter. Gathering insights and making them available to customers is the ML business model. Who cares what language you use as long as you know how to implement the actual concepts.
Most with Erlang? The same Erlang that has explicit concurrency and parallelism? The same Erlang that you have to understand the difference between concurrency and parallelism to use on anything bigger than Hello World? That Erlang?
Some of the guys I work with use cython for compiled stuff. It's not the best solution but it does help with some of the issues brought up. Y the python haters
Different people have different definitions
Buuuulshit. Words mean what they mean. *Especially* technical words.
Ezekiel 23:20
Well, what you need is C (or C++). And if you want some sort of sane layer beyond that, then there's PHP and PHP extensions. PHP 8's roadmap is being defined *right now* and there's talk of a JIT for CLI and solid multithreading support. So there you go.
(How does Faceplant's "chief AI scientist" not know about the changes being made in PHP core when they run HVVM on their entire infrastructure?!)
You're full of bullshit. If you knew the difference you would have said it, instead you're wasting words. Wasted words is the definition of bullshit.
"First they came for the slanderers and i said nothing."
... just set up some deep learning tools in that new language that are:
(1) feature rich
(2) have good mechanisms for separating and visualizing data and
(3) have been well vetted by industry.
I'll hold your beer.
This.
Garbage code is garbage code, and has very very little, if anything, to do with the language it is written in.
It is just as easy to write garbage C++ as it is garbage python - perhaps even easier.
Stop saying stupid things and say what the difference is between parallelism and concurrency. Concurrency is really nice in Erlang but I was more attracted by its stability.
"First they came for the slanderers and i said nothing."
Stop saying stupid things.
Right back atcha.
what the difference is between parallelism and concurrency.
Maybe you should read the Getting Started in Erlang guide. Sections 3.1 to 3.3 deal with concurrency; 3.4 deals with one off the many forms of parallelism. But, of course, you already knew that, right? Your claim to Erlang wasn't just BS, right?
I was more attracted by its stability.
Sure you were, sweetheart. Robustness is the chapter after Concurrency in the Getting Started guide. You know why? Because concurrency and parallelism is how Erlang achieves robustness.
The link you gave doesn't define the difference between concurrency and parallelism. I'm not sure you read it.
"First they came for the slanderers and i said nothing."
Btw if your point was that you think Erlang has better concurrency than C#, you should have just said that. You might be right.
"First they came for the slanderers and i said nothing."
If you need it spelled out, concurrency happens inside a task, e.g. mullti-threading. Whereas parallelism is multiple tasks, possibly on multiple machines. Erlang is better than C# at both because it was designed specifically to do them well.
Some elementary Googling turned up a link for your elementary computer education.
Even you will understand the difference after you have read it.
https://howtodoinjava.com/java/multi-threading/concurrency-vs-parallelism/
Btw if your point was that you think Erlang has better concurrency than C#, you should have just said that. You migh
"First they came for the slanderers and i said nothing."
You're supposed to have already seen this a looong time ago. The fact that you haven't is not my problem.
Ezekiel 23:20