Slashdot Mirror
Deep Learning May Need a New Programming Language That's More Flexible Than Python, Facebook's Chief AI Scientist Says (venturebeat.com)
Deep learning may need a new programming language that's more flexible and easier to work with than Python, Facebook AI Research director Yann LeCun said today. From an interview: It's not yet clear if such a language is necessary, but the possibility runs against very entrenched desires from researchers and engineers, he said. LeCun has worked with neural networks since the 1980s. "There are several projects at Google, Facebook, and other places to kind of design such a compiled language that can be efficient for deep learning, but it's not clear at all that the community will follow, because people just want to use Python," LeCun said in a phone call with VentureBeat. "The question now is, is that a valid approach?" Further reading: Facebook joins Amazon and Google in AI chip race.
What we need is less Facebook, less Google, less Amazon.
Just use C++
or perhaps prolog...
*** Suerte a todos y Feliz dia!
I have one character for you ")"
Unfortunately Procedural and Object Oriented languages, have gotten a foot hold, and training someone to code functionally is quite difficult, and for parallel processing it may be even trickier.
If something is so important that you feel the need to post it on the internet... It probably isn't that important.
or ruby..
If indeed f "deep learning" has hit the wall with current languages.... perhaps the hardware itself in it's current configuration isn't up to the task.
Since all languages simply compile to machine code, which runs on an OS making hardware calls, a top level restriction implies a deep level constraint.
To develop the next -real- AI, requires a different hardware approach running some as of yet unconvinced OS - followed by a language.
... for building Skynet, and it'll be Lisp or perl.
And we all know which one the Lord used: https://xkcd.com/224/
Hire a Linux system administrator, systems engineer,
Nuf said.
Over the past few months I have spoken to a few recruiters I know who were asking me to give them all the senior C++ engineers I know or if I personally was interested.
In what?
Doing complete rewrites of giant mountains of garbage Python code written by twenty something year old hipsters or older researcher type people.
It is boring as fuck work but companies and organizations are desperate and willing to pay huge amounts of money to rid themselves of the clusterfuck that is Python.
I think it fits the requirements quite well.
That's because he isn't sure yet.
Just because you are unhappy with their current tools and know through decades of experience that "more of the same" is not going to improve things, that doesn't mean you know what the future should look like.
sub f{($f)=@_;print"$f(q{$f});";}f(q{sub f{($f)=@_;print"$f(q{$f});";}f});
it's called Common Lisp.
I am picturing the current crop of 'the past was stupid, never learn from it!' programmers diving into LISP, Smalltalk, or Prolog.
But as always, designing new languages is more fun than learning old ones, and learning from past mistakes makes you uncool.
Everyone is apparently using an interpreted language? Sounds to me like they need to apply the second word of A.I. , since they keep insisting on calling it that.
LISP, then?
I've abandoned my search for truth; now I'm just looking for some useful delusions.
"Whenever somebody's conclusion is that we need a new programming language, I know we are dealing with an idiot or more of a philosopher than a programmer."
And yet the idiots/philosophers gave us every languge we use to today. There's a reason we're not writing in machine code.
SJW n. One who posts facts.
Don't use languages designed by language lawyers with esoteric constructs designed to lower productivity of average developers.
A new language is not needed, it's a call to ....
- fund a large project to design a language
- get lots of hype, sell lots of newsprint, magazine space and web blog advertisements,
- give lots of experts speaking fees
- sell lots of training courses online and in person
- sell books
- sell tool sets
- let people attend experts conferences
- help increase the GDP
- provide jobs for PhD holders to research it
-
A language is written for the reader and not the elegance i wet my diaper cool shiny needs of the original developer.
Programming 101 from the 1960s:
"How hard is it extract the business intent of a program by a developer unfamiliar with the system, business processes and computer architecture?"
Programming is not creative writing elegance is not the top goal in computer code, program structure or language.
Another to file in the it's lisp, lisp is good, lisp is the best VS we need to read the program and not be an uber-expert in it.
Machine code is too fast.
A hundred years ago it took five floors (200 per floor) of accountants labouring 10 hours a day more than 6 months (with no days off) to make the same fuck-up as your Python code did in just 2 seconds. Of course, your machine code could have done that same fuck up in less than a millisecond, which is far too fast. That is why we do not use it.
Functional is the easiest code for parallel processing.
Nah, C# with they yield style of automatically doing multi-threading is the easiest.
Why do you think it has been making such a come back?
It hasn't. Some of the nice functionality has been imported into Java, but hardly anyone is talking about immutability.
"First they came for the slanderers and i said nothing."
That's definitely what we need. A new programming language.
We suffer more in our imagination than in reality. - Seneca
I don't know, because I'm the exact opposite of Yann LeCun: he's an AI guy unsatisfied with programming languages, and I'm a programming language guy unsatisfied with AI.
I haven't worked in deep learning, so I don't know if he has well-formed complaints or he's just generally unsatisfied.
sub f{($f)=@_;print"$f(q{$f});";}f(q{sub f{($f)=@_;print"$f(q{$f});";}f});
I mean, sure, a single data processing pipeline might have to use 6 different conda environments, each with different dependencies and python versions due to tool and libraries are often deprecated with even minor point changes to python versions...... oh yeah, all that and then you have to shoe-horn in tensorflow (or something else).
you have never done any serious work in a functional language or any serious parallelism.
I've done both.
"First they came for the slanderers and i said nothing."
We heard these exact same people complain about other languages in the 1980s. They complained about Lisp, then Scheme, then Haskell. No one is forcing them to use them, and no one is forcing them to use Python. A General Purpose AI is a pipe dream that will not ever come to fruition, no matter what language you are using. Deep Learning is stuck again because there is no fundamental theory of learning. We have no idea how we learn. Neuroscience will probably discover that learning is a very biological trait, not one that can be copied or simulated in discrete mechanical systems.
Insert a bunch of NOPs in the loop to give the port time to catch up...
Seven puppies were harmed during the making of this post.
If someone develops a language that offers true leverage over Python then the transition *will* happen and all that will be left unanswered is "how long will it take for the transition to happen?"
Guns don't kill people; Physics kills people! - John Lithgow as Dick Solomon on Third Rock From The Sun
I say this every other day in one form or another, but let me try again: we all keep jumping up and down and shouting about whether we need a more flexible language or a fussier language with super-strong type-checking and encapsulation and what not, and some are deeply convinced that Pythons's fascist attitude toward formatting is excellent, and some are not -- one of the reaons the Go project was started, from what I understand, was to get away from syntactic whitespace--
None of us really know who's right about any of this, because to really find out someone would need to do experiments, and designing these experiments would require knowledge of social science techniques, and "Computer Scientists" refuse to consider doing that kind of science, because most of them are really mathematicians in disguise.
So we're stuck with a religious fervour for concepts like "elegance" without any real reason to believe any of it is right.
Norvig doesn't laugh.
Ezekiel 23:20
I have one character for you ")"
That's not a character, that's a SIMPLE-STRING of length 1. ;)
Ezekiel 23:20
Nah, C# with they yield style of automatically doing multi-threading
I know next to nothing about C#, but I was under the impression that this was a concurrency mechanism - basically a control flow construct - not a parallelism feature. Did something change?
Ezekiel 23:20
I agree with Yann that our current crop of languages aren't well suited to deep learning, but I'm not sure it's a Python specific problem. I don't think whitespace, threading, syntax etc are the barrier.
It's much more that Python is fundamentally an imperative language, and deep learning doesn't fit into either the imperative or functional category, I really think DL deserves its own category, designed from the ground up for manipulating tensor data structures of unknown shapes.
I haven't come across any language that encapsulate strong typing for tensor shapes, so you end up tracking shapes with pen and paper while debugging your code (and not everything is caught at compile time). Named tensors would be a good start (more information here http://nlp.seas.harvard.edu/Na...) but that wouldn't solve everything. People can't agree on computation graph vs a forward/back functional approach - PyTorch prefers the latter, TensorFlow originally preferred the former (but now seems to be migrating to the latter).
Julia does seem to be the most promising, but that may just be "the grass is always greener" speaking. I haven't played with it myself.
... there was a programming language designed specifically for that
I'm not convinced that there is a meaningful difference between concurrency and parallelism.
"First they came for the slanderers and i said nothing."
Yep. The runtime starts main(), and returns the return value of main to the OS(). So basically it does START and END.
Everything in between is the responsibility of main(). It needs not interact with the OS at all, other than being started by the OS and telling the OS when it's done. Only for standalone programs, though - kernel modules don't need to do those two things.
There is a very useful kernel module which does nothing but allocate some memory. That's all it does. :)
It's used when you have a a few bytes of dodgy memory. You have that module allocate those memory addresses for it's use - and then do nothing else.
To quote Albert van der Horst (Feb 17 '14 at 13:23) "I would rephrase the question as "what features of a language are necessary for hard AI?" This is a valid and useful question, and the outcome will be that only Lisp, Forth & assembler have what it takes to do hard AI. – "
...seriously?
Ezekiel 23:20
Not Invented Here syndrome at work?
People want to keep using Python, it must be doing something right.
On a long enough timeline, the survival rate for everyone drops to zero.
You never know, right?
> There is definitely an expectation of some basic facilities available to your code.
What do you have in mind when you say "some basic facilities"?
Yeah. Different people have different definitions and from the point of view of the programmer it doesn't really matter if they are happening at the same time or not.
"First they came for the slanderers and i said nothing."
Lisp, Erlang, Haskell. Most with Erlang, least with Haskell. Some people call Python a functional programming language, but it's not really.
"First they came for the slanderers and i said nothing."
Different people have different definitions
Buuuulshit. Words mean what they mean. *Especially* technical words.
Ezekiel 23:20
You're full of bullshit. If you knew the difference you would have said it, instead you're wasting words. Wasted words is the definition of bullshit.
"First they came for the slanderers and i said nothing."
... just set up some deep learning tools in that new language that are:
(1) feature rich
(2) have good mechanisms for separating and visualizing data and
(3) have been well vetted by industry.
I'll hold your beer.
This.
Garbage code is garbage code, and has very very little, if anything, to do with the language it is written in.
It is just as easy to write garbage C++ as it is garbage python - perhaps even easier.
Stop saying stupid things and say what the difference is between parallelism and concurrency. Concurrency is really nice in Erlang but I was more attracted by its stability.
"First they came for the slanderers and i said nothing."
The link you gave doesn't define the difference between concurrency and parallelism. I'm not sure you read it.
"First they came for the slanderers and i said nothing."
Btw if your point was that you think Erlang has better concurrency than C#, you should have just said that. You might be right.
"First they came for the slanderers and i said nothing."
Btw if your point was that you think Erlang has better concurrency than C#, you should have just said that. You migh
"First they came for the slanderers and i said nothing."
You're supposed to have already seen this a looong time ago. The fact that you haven't is not my problem.
Ezekiel 23:20
Fundamentally the same problem as with Python but worse due to weak software engineering.
--- widget evolution: enhanced, plus, super, ultra, extreme, exxxtreme, ultra-extreme,