Slashdot Mirror
Deep Learning May Need a New Programming Language That's More Flexible Than Python, Facebook's Chief AI Scientist Says (venturebeat.com)
Deep learning may need a new programming language that's more flexible and easier to work with than Python, Facebook AI Research director Yann LeCun said today. From an interview: It's not yet clear if such a language is necessary, but the possibility runs against very entrenched desires from researchers and engineers, he said. LeCun has worked with neural networks since the 1980s. "There are several projects at Google, Facebook, and other places to kind of design such a compiled language that can be efficient for deep learning, but it's not clear at all that the community will follow, because people just want to use Python," LeCun said in a phone call with VentureBeat. "The question now is, is that a valid approach?" Further reading: Facebook joins Amazon and Google in AI chip race.
What we need is less Facebook, less Google, less Amazon.
True, but even C/C++ and Assembly doesn't provide an "easy" way to do threading, which is the issue.
Scripting languages, basically do not do threading, of any kind, at all. They're too slow to synchronize across threads, which makes invoking threads inside them fruitless.
While C is ultimately the right language to do everything in (not C++) , the real issue is that cpu's are expanding in cores, just like GPU's have, yet GPU's have standardized more or less on just three API's, OpenGL, Direct3D and Vulkan. So if you can write a program against Vulkan, you have as close to bare hardware as you are going to get. But for CPU's, there is still a 57 flavors of rubbish programming languages and no standard runtime that works for all of them, at best most of these programming languages are still developed in C or C++ and thus require a complete C AND C++ runtime to function.
Python is not written in Python. Java is not written in Java. If a language can not compile itself, it's not flexible enough to be used for any of the three main corner stones of software development: Operating Systems, Applications, and Games. While you certainly can write an application or game with a scripting language, it will be slow, it will be limited by the operating system's own libraries (eg 32-bit libraries on a 64-bit OS as just one example) and generally require more maintenance than simply writing it in C to begin with.
Memory overflow errors are caused by people learning programming languages like Java first instead of C, because if you learn C first, you then learn how to initialize memory, and how big memory chunks actually are.
Over the past few months I have spoken to a few recruiters I know who were asking me to give them all the senior C++ engineers I know or if I personally was interested.
In what?
Doing complete rewrites of giant mountains of garbage Python code written by twenty something year old hipsters or older researcher type people.
It is boring as fuck work but companies and organizations are desperate and willing to pay huge amounts of money to rid themselves of the clusterfuck that is Python.
I think it fits the requirements quite well.
True, but even C/C++ and Assembly doesn't provide an "easy" way to do threading, which is the issue.
Nothing is easy to multithread in because multithreading anything more than the most basic of processes is inherently complex. This doesn't apply to SIMD structures which is the main reason why GPU's can be so parallel. One pixel doesn't care what color the other one is.
While C is ultimately the right language to do everything in (not C++) , the real issue is that cpu's are expanding in cores, just like GPU's have, yet GPU's have standardized more or less on just three API's, OpenGL, Direct3D and Vulkan. So if you can write a program against Vulkan, you have as close to bare hardware as you are going to get. But for CPU's, there is still a 57 flavors of rubbish programming languages and no standard runtime that works for all of them, at best most of these programming languages are still developed in C or C++ and thus require a complete C AND C++ runtime to function.
A lot to disect here. So for starters APIs and languages aren't the same thing and the main graphics APIs can be interfaced with multiple languages, I've seen programs written in everything from Javascript, Python, C, and Java interface with the OpenGL api. Second, there is no such thing as a C/C++ runtime. Runtimes only exist in interpreted languages like JS with its DOM or in ones that compile to byte code for a nonexistent VM like Java. C/C++ compile to binary for a given architecture. That's the fundamental difference between compiled and interpreted languages.
Python is not written in Python.
Actually, some runtimes are.
If a language can not compile itself, it's not flexible enough to be used for any of the three main corner stones of software development: Operating Systems, Applications, and Games.
I don't know about flexibility, one of the biggest strengths of these dinky languages is their flexibility. Their biggest weakness however is a lack of efficiency and performance. If you had said they are unsuitable due to performance issues or an inability to run direct on the metal then I would agree with you. It's impossible to write an OS kernel that runs on a real machine by itself in javascipt or python. It cant be done because they both require JITs to work.
While you certainly can write an application or game with a scripting language, it will be slow,
Not a guarantee, but likely depending on complexity. There have been many successful games written in Java, minecraft for example.
it will be limited by the operating system's own libraries (eg 32-bit libraries on a 64-bit OS as just one example) and generally require more maintenance than simply writing it in C to begin with.
Eh, any program referencing external libraries has this problem. See issues with old C++ programs referencing deprecated Win32 APIs trying to run in Windows 10. However it is possible with some careful coding and luck to write a complex application that can work for decades unmodified in C. The same cannot be said for JS. If you write something complex in JS and dont touch it, three years later it wont work. This is especially true if you use some idiotic technology like NPM. Because storing your dependencies in the cloud is a great idea.
Memory overflow errors are caused by people learning programming languages like Java first instead of C, because if you learn C first, you then learn how to initialize memory, and how big memory chunks actually are.
While I do appreciate C, I think you give it too much credit. For one any serious Java programmer has to learn memory management eventually because the built in garbage collection is trash and once you get to a certain complexity level its no longer good enough. On top of that C's memory model is not an accurate representation of how a computer's memory model works anyways. I think this is one of C and C++'s biggest weaknesses and contributes to many of the mistakes programmers make in regards to memory management when using them.
"Whenever somebody's conclusion is that we need a new programming language, I know we are dealing with an idiot or more of a philosopher than a programmer."
And yet the idiots/philosophers gave us every languge we use to today. There's a reason we're not writing in machine code.
SJW n. One who posts facts.
Threading in ANSI C is pretty straightforward. There are inherent complexities in multithreaded code, but those can't be ignored by any language. In the context of what C code looks like, I think the threading interface is about as simple as you can get. Do you have any examples of a simpler model?
We heard these exact same people complain about other languages in the 1980s. They complained about Lisp, then Scheme, then Haskell. No one is forcing them to use them, and no one is forcing them to use Python. A General Purpose AI is a pipe dream that will not ever come to fruition, no matter what language you are using. Deep Learning is stuck again because there is no fundamental theory of learning. We have no idea how we learn. Neuroscience will probably discover that learning is a very biological trait, not one that can be copied or simulated in discrete mechanical systems.
To recap: The first comment mentioned the C/C++ runtime, i.e. the distributable implementation of the standard, providing things like malloc. The second comment misinterpreted this to mean a runtime environment, e.g. the java executable, or the .NET common language runtime (CLR). The third comment noted that the second comment was mistaken, and once again referred to the runtime. Then your comment again mistakes that for runtime environment
Conclusion: Words are hard.
Captcha: obvious.
You are approaching the problem in the wrong way.
You capture the problem with this line "If more than one thread is trying to access the same resource at the same time the headaches begin".
By far the best solution is to avoid this issue in the first place. I.e. write in a functional style, prefer immutability over mutability, and don't share mutable state in concurrent code. Then you will find that reasoning about concurrent code is much much easier.
Of course, C doesn't really support this paradigm, which is why having written in C for 20+ years I think that it is a poor language.
Every time I see another bug due to code written in C or C++ due to a buffer overrun, I let out depressed sigh, because I know that someone who is either arrogant or inexperienced is going to stay that the problem is with the programmer rather than the language.