Slashdot Mirror
Deep Learning Is Eating Software (petewarden.com)
Pete Warden, engineer and CTO of Jetpac, shares his view on how deep learning is already starting to change some of the programming is done. From a blog post, shared by a reader last week: The pattern is that there's an existing software project doing data processing using explicit programming logic, and the team charged with maintaining it find they can replace it with a deep-learning-based solution. I can only point to examples within Alphabet that we've made public, like upgrading search ranking, data center energy usage, language translation, and solving Go, but these aren't rare exceptions internally. What I see is that almost any data processing system with non-trivial logic can be improved significantly by applying modern machine learning. This might sound less than dramatic when put in those terms, but it's a radical change in how we build software. Instead of writing and maintaining intricate, layered tangles of logic, the developer has to become a teacher, a curator of training data and an analyst of results. This is very, very different than the programming I was taught in school, but what gets me most excited is that it should be far more accessible than traditional coding, once the tooling catches up. The essence of the process is providing a lot of examples of inputs, and what you expect for the outputs. This doesn't require the same technical skills as traditional programming, but it does need a deep knowledge of the problem domain. That means motivated users of the software will be able to play much more of a direct role in building it than has ever been possible. In essence, the users are writing their own user stories and feeding them into the machinery to build what they want.
Nom nom nom
Deep learning's eating software and software's eating the world. We just need a few waves of Chinese needle snakes to eat Deep learning. Then gorillas to eat the snakes. Finally when wintertime rolls around, the gorillas simply freeze to death.
Hi! I make Firefox Plug-ins. Check 'em out @ https://addons.mozilla.org/en-US/firefox/addon/youtube-mp3-podcaster/
Ya, I'm calling BS. Give us some concrete examples of how ML/AI/DL is doing anything other than burning CPU cycles on public clouds that drive up revenue for the cloud vendor.
Alan Bradley: Some programs will be thinking soon.
Dr. Walter Gibbs: Won't that be grand? Computers and the programs will start thinking and the people will stop.
Most software today and in the coming decades is designed and developed to support business processes or data flow and execution in scientific processes. These systems need a deterministic and foreseeable behavior. Yes, you may use "learning" classification mechanisms such as neural networks to support some tasks, but this is not changing how we develop software. Especially, developing software is usual a technical and social process, as you have to understand the demands and needs of users, which require interviews and discussions with users. You also need to communicate with UI designers to develop together with users and UI designers useful and easily to understand interfaces. And yes, you have to map all this onto technology.
"how deep learning is already starting to change some of the programming is done."
Perhaps how some of the English is done too?
This is marketing dribble from some guy at some company, neither of which are relevant. This is the kind of blogspamvertisement I'd expect in my inbox after a sales/marketing-oriented PM got a bug up their ass to research something outside their realm of expertise, not on /.
Except for the fact the deep learning systems, may not be efficient enough. There are some tasks which it excels at, however some tasks there just isn't the volume or rewards for outcomes for the system to adapt fast enough to.
Lets say a flood control dam system. Which is is strictly coded if water goes above this line, open the dam and flood the down, because otherwise the damage is far greater than flooding the town. The reason why this is computer controlled is because it takes the faulty morals out of the picture. You wouldn't want to be the one to press the button and flood the town, even though it is for the greater good. A Deep Learning system, will want to take more variables into account, and will either decide that it is important to press the button before it hits the limit, as it would only create minor flooding problems, or too late figuring that it can wait a little longer creating a catastrophic failure.
Deep Learning often creates a lot of superstitious like behaviors (As shown in the funny walk of Googles Deep learning to walk video) Where it learned to do the task, but not at optimal way. This may just be a mater of time for the technology to get better, however it isn't a time to panic and toss out your CS Degree and get a job at your local Walmart as a greeter.
If something is so important that you feel the need to post it on the internet... It probably isn't that important.
I'd be willing to speculate that the underlying sad story is one where magical black boxes, in all their imperfection, can still do better than e.g. run of the mill application software written by the majority of programmers who got their degrees in the last decade or so. Not in things like bookkeeping of course, what with tax codes and all, couldn't learn that by example if one tried -- but for genuinely nebulous things like individual preferences in conference room scheduling, or other frankly shithead jobs.
However, if this were to happen, we'd run out of juniors; or everyone entering the line of work would have to do said low-status jobs into a shoebox just to get to the level that black boxes can't reach.
ML is generally enabling scenarios that were just too tedious to actually do by developer hands. Sure there are specific scenarios where developers had done the best they can (and generally failed) with hopelessly unstructured data, but for the most part those problems were just left untouched as infeasible to do manually.
For the vast majority of software development, ML doesn't add anything. If you have no unstructured data or a way to impose structure, ML doesn't do anything over boring old programming. Even when you find yourself in one of the very chaotic, large, and diverse data sets where ML can in theory help you sort through, you have to first chew through enough data in training to get decent confidence. So you not only need a large data set, you also need to have a continued need after human assisted training has already done the work on a big chunk of that data. Even then you may be grasping for some intelligent way to apply ML techniques, because the kicker is you have to have some sort of real idea of what to do, even if you have a 'how to do it'.
Big Data has done this same song and dance. ML is now the purported answer to 'once collected and have tools to analyze, most orgs have no idea what to do with the data'. I suggest that the orgs will still have no idea what to do with the data, and ML won't move the needle much in the wider market because the root cause is just a general lack of thoughts on what to do with the data. This is the curse of hyped adoption, the vast majority of adopters will be disappointed because it doesn't magically solve.
XML is like violence. If it doesn't solve the problem, use more.
These systems will not gain the insight and expertise needed for many areas that require real-time responses. They may produce wonderful results in predicting stock market prices from historical data, but they will take far too long to be useful for the microsecond resolution trading that is done today. That takes teams of people designing new hardware and programming it, plus real-world factors such as proximity to the stock market's computers. As tech advances, the already trained, overly complex systems will need to be updated and re-trained. There are many reasons why the real-world usefulness of these programs will be limited, despite the starry eyed predictions of their heralds.
On the one hand you take life too seriously, and on the other, you do not take playful existence seriously enough. Seth
And by "deep learning" in most cases they mean "linear regression on cleaned-up data"
-------
1. Enjoy your job
2. Make lots of money
3. Work within the law
Choose any two.
Once upon a time, I did my doctorate in machine learning. The machines were less powerful, but the algorithms? Basically the same as they are today. Sorry, the stuff most widely in use is still the same back-propagating neural networks. The machines are just faster, so the networks can be bigger. That's it.
Neural networks can work really well on specific problem domains. The problem is: You have no idea what they are actually learning. The features that a network identifies within its layers are not really accessible to us. The problem lies, imho, in the total lack of domain knowledge. Since the network doesn't understand what the objects in those pictures are, they are doing a purely mechanical analysis of some (and who knows which) aspects of the pictures. They can learn some really weird things.
In a well-trained network, the results mostly coincide with our expectations. In a completely isolated domain, like chess or go, a network can be trained sufficiently to perform quite well. However, in open domains, they are fragile: we have no idea when they will break. Look at the video of the turtle being identified as a rifle (in the link above). Why does the identification jump seemingly at random? When will a cat will suddenly be guacamole? When will a pedestrian crossing the road will suddenly be just a pile of leaves? We have no idea, none.
It is certainly true that selecting and managing training data is a very different task from classic programming. However, it doesn't really take much domain knowledge. In most domains, gathering training data is tedious, not difficult. The hard part comes in figuring out how to make the best use of that data to train and test a network - and that requires a deep understanding of how the neural networks work (and how they don't work). Plus, frankly, a huge pile of trial and error, because there aren't many rules on how to best structure a net for any particular task.
Enjoy life! This is not a dress rehearsal.
If so, at least the article's jumble of catchphrases still moved the universe forward. Yay.
...when you can input a photograph of an airplane and the Navier-Stokes equation, and get a flight simulator as output.
existing software project doing data processing using explicit programming logic, and the team charged with maintaining it find they can replace it with a deep-learning-based solution
So, that team relied on a pre-made code to deal with a specific part of the implementation! Why is this news? Although I personally prefer to develop most of the (data-management) algorithms myself, relying on third-party dependencies is a quite common and acceptable proceeding in software development.
Custom Solvers 2.0 = Alvaro Carballo Garcia = varocarbas.
Most programming jobs involve connecting stuff together. Converting a database format to another, design a GUI around it, add the entry points to turn it into some kind of module, extract or integrate features, etc... Even machine learning typically involves gathering a bunch of data turn it into a form that's acceptable for the learning module and feeding the results to some other component.
I don't know how machine learning will help with all that stuff. An AI won't write a video game, it can help making mobs smarter, generating convincing maps or optimizing revenue. But in the end that's just a module connected to other modules, and programmers will be needed to put the round peg into the square hole.
It will make things a bit more high level, as always. But except for a bunch of PhDs, I don't expect major changes in the way people program.
Except for the fact the deep learning systems, may not be efficient enough. There are some tasks which it excels at, however some tasks there just isn't the volume or rewards for outcomes for the system to adapt fast enough to.
That sort of goes out the window once the Singularity hits, though. A sufficiently advanced AI with the ability to goal-seek improvements to itself will wrap its own subsystems in learning simulators. It won't need to train on real-world disasters, it will simply iterate on simulated ones and then iterate on updating the simulation. Meta-learning, if you will.
The Go AI, which is now approximately 100 times smarter than it was before and is becoming nearly impenetrable for Go experts to understand how/why it's making the decisions it is, is a prime example of this. After a certain point, its own mechanism of adaptation outpaces our own.
Hire a Linux system administrator, systems engineer,
Wait, you mean Walmart pays those people to do that? I thought they were just deranged people with nothing better to do!
#DeleteFacebook
The lesson here is that you should have learned the languages of the future: Turbo-Pascal and Fortran.
#DeleteFacebook
No it's not, the value of BTG is crashing.
#DeleteFacebook
I work on mostly CRUD and e-reporting applications. Generally an org wants these kinds of apps to be predictable and reliable, not "organic" (trial and error). I don't see organic learning as a viable way to program such in the future.
However, I can see AI being used to test the apps and find potential bugs in the source code, being that "suspicious pattern detection" is something it can do relatively well. It may also suggest code, schema, and UI refactorings. But such AI would be an adviser to programmers, not behind the wheel itself.
Clippy impression: "It looks like you have a lot of code pattern duplication between modules 7, 22, 43, and 51. Would you like help refactoring?"
Table-ized A.I.
You must be mistaken here, I think it was the rustmonster eating it all up. Only the blessed +2 rustproof vorpal blade can help you now, you lowly programmer. Tensorflow has now ascended.
This sounds suspiciously like a lot of 4GL promises that were made in the 80s and 90s. They also sound like the kinds of promises made by Microsoft promoting their distributed data model based on Office. Many times I've seen users get in over their heads with systems that start out easy, but get complicated quickly. Worse, sometimes they ended up with processes that produced erroneous data. Ultimately, they resort to piling the whole smoldering hot mess onto the programmers, who have to "make it work" somehow.
Proverbs 21:19
For computational linguistics (translation, analysis, etc), machine learning is not a net gain. What ML proponents forget to factor in is the vast time spent on gathering and hand-annotating large quantities of text (gold corpora).
Even worse, for many many languages, these gold corpora simply do not exist and there are no plans on making them, or they are too small to be used for ML.
And even when the gold corpora do exist, models trained on them become tightly coupled with the data. They become domain specific. In order to escape domains, you need an order of magnitude more data.
Instead, one can make a domain-independent rule-based system in a fraction of the total time spent on machine-learning models. But rule-based has become this weird anathema - people will even write papers that use rule-based methods, while hiding it behind machine-learning terms.
I'm sure this also holds for other fields.
It is: https://en.wikipedia.org/wiki/...
Confucius say, "Find worm in apple - bad. Find half a worm - worse."
In my line of work I use a lot of mathematical optimisation. As Stephen Boyd says in his course, everybody working in optimisation has at some point this epiphany: "everything is an optimisation problem". And this is true. However to make it work you need to be very good at mathematical modelling, you need to know your methods, and most of the time the problem is unsolvable anyway by the classic methods.
In this instance maybe a lot of programming can be modelled by some deep NN. However you have to come up with a relevant architecture for your problem, you need to train it, and you need to evaluate it. It may save you time to do so, but if you need so solve something like FizzBuzz, that may not be the best way.
Oh dear God please no. The Basic, PHP and Microsoft paper admins who can't program their way out of a paper bag will now all become Subject Matter Experts on everything to help us all out.
"Why does that go there? Things doesn't work unless it does." "Wha's your problem, bud? Those leaks in the damn have always just been there, don't worry about them."
And you thought things were bad now -- just wait until NO ONE knows what's actually going on, they only know what's SUPPOSED to be happening.
Of course with all of the multitude of languages, support libraries, and computer inter-connects maybe we've already got that now.
------
What if the light you see when you die is the headline of an oncoming train?
If the universe is someone's simulation -- does that mean the stars are just stuck pixels?
Public Service Announcement: deep learning is not even Turing complete. It is simply fancy nonlinear regression that works well on hierarchically-ordered domains.
Not only eating software, it's eating disk like crazy. I've seen millions poured into this deep learning stuff at a governmental level and after a year of buying expensive servers with lots of CPU and lots of memory and lots of NVidia drivers running Linux with even big frickin' Oracle databases or hadoop we get - TRASH!
Sounding more like it did in the 1980s... bit promises... short on reality.