Slashdot Mirror
Deep Learning May Need a New Programming Language That's More Flexible Than Python, Facebook's Chief AI Scientist Says (venturebeat.com)
Deep learning may need a new programming language that's more flexible and easier to work with than Python, Facebook AI Research director Yann LeCun said today. From an interview: It's not yet clear if such a language is necessary, but the possibility runs against very entrenched desires from researchers and engineers, he said. LeCun has worked with neural networks since the 1980s. "There are several projects at Google, Facebook, and other places to kind of design such a compiled language that can be efficient for deep learning, but it's not clear at all that the community will follow, because people just want to use Python," LeCun said in a phone call with VentureBeat. "The question now is, is that a valid approach?" Further reading: Facebook joins Amazon and Google in AI chip race.
"Whenever somebody's conclusion is that we need a new programming language, I know we are dealing with an idiot or more of a philosopher than a programmer."
Second, there is no such thing as a C/C++ runtime.
Yes, that thing called crt0 that you've seen all your life is an illusion.
On a modern CPU, the C runtime doesn't have to do much. It has to set up the stack, argc/argv/envp, atexit handlers, and a few more random things. But it very much exists.
Also consider that C compilers are allowed to generate calls to memcpy if you assign a large struct or something, and many of them do.
sub f{($f)=@_;print"$f(q{$f});";}f(q{sub f{($f)=@_;print"$f(q{$f});";}f});
You can absolutely use Julia productively without getting into all the extra stuff. You can write code similar to Matlab or Numpy. Later, when you want more performance, you can delve into types more. Admittedly, the documentation emphasizes the multiple dispatch sophistication, and maybe Julia has a longer on-ramp than Python. In the past Julia was evolving very quickly, but now that 1.0 has been released, you can stick with that. But there is no other new language that has as strong a community dedicated to readable, powerful high-performance numerics. And the appeal of Julia is not that it does what currently Python or R can, but it's a better place for libraries to written by experts in the language itself. I can't think of a better language for doing research in numerical optimization, when you're really exploring new ideas and not just plugging into someone else's canned, but confining, "solutions". Most Python numerical libraries must, for performance, ultimately rely on C or C++ underneath, so becoming expert at Python does not help you in contributing to new high performance libraries. By contrast, high performance libraries for Julia can be written in Julia itself, so therefore Julia can be a very good long term investment. Please, tell me what high performance Python numerical libraries are written in Python, without C or C++?
Julia has a tremendous problem. It's not designed for users, it's designed for Julia designers.
I have to disagree with that. Julia is designed for users, but it knows that its use case is not Python's use case.
Julia was designed as an upgrade for Fortran programmers. Like all good upgrade languages, it learned from all of the languages which tried (and failed) to replace Fortran previously, like SISAL and Fortress.
There is a cohort of programmers for whom "the standard approach" means Python's highly idiosyncratic approach. In my (admittedly limited) experience, anyone who predates the rise of Python tends to have no problem picking up Julia.
sub f{($f)=@_;print"$f(q{$f});";}f(q{sub f{($f)=@_;print"$f(q{$f});";}f});
There is no easy way to do threading, by its very nature. It's not a language thing, it's a computer thing. If more than one thread is trying to access the same resource at the same time the headaches begin: who goes first, how do you get the threads to accept its place in the queue and "know" when the other is done, etc, etc, etc. Languages that support memory locking of course make things easier but you still have to think the program through very clearly and often you end up with rather unusual and not reproducible bugs.
However I posit that many people who use higher level languages have no actual idea of how a computer works or what they are actually doing, unlike us old timers who grew up with assembler.
Seven puppies were harmed during the making of this post.
Scripting languages, basically do not do threading, of any kind, at all. They're too slow to synchronize across threads, which makes invoking threads inside them fruitless
Bullshit. You CAN do asynchronously loading of assets with JavaScript.
Asynchronous and multithreaded programming are two different things. Concurrency is NOT the same thing as parallelism.
Ezekiel 23:20
The following is the C runtime, crt0. It is 9 lines of assembler: .text .globl _start
_start: # _start is the entry point known to the linker
mov %rsp, %rbp # setup a new stack frame
mov 0(%rbp), %rdi # get argc from the stack
lea 8(%rbp), %rsi # get argv from the stack
call main # %rdi, %rsi are the first two args to main
mov %rax, %rdi # mov the return of main to the first argument
call exit # terminate the program
Compare the millions of lines in a the Java runtime.
C has a runtime like Wiz Khalifa has boobs.
To a compiler writer (which is where I got my start) a compiler's runtime is any code that is needed to run a program but isn't generated by the compiler when an end user compiles their program.
C runtimes used to be a lot bigger than they are now. In the days of MS-DOS, you couldn't assume the presence of a FPU, so floating point was often compiled into calls into the FP runtime. Even today, microcontrollers often don't have instructions which directly implement basic C operations (e.g. 64 bit integer division) so these operations are typically compiled as calls to runtime routines.
As CPUs get more powerful, C runtimes get smaller. But to say there's no such thing is flatly untrue.
sub f{($f)=@_;print"$f(q{$f});";}f(q{sub f{($f)=@_;print"$f(q{$f});";}f});