Slashdot Mirror
Deep Learning Is Eating Software (petewarden.com)
Pete Warden, engineer and CTO of Jetpac, shares his view on how deep learning is already starting to change some of the programming is done. From a blog post, shared by a reader last week: The pattern is that there's an existing software project doing data processing using explicit programming logic, and the team charged with maintaining it find they can replace it with a deep-learning-based solution. I can only point to examples within Alphabet that we've made public, like upgrading search ranking, data center energy usage, language translation, and solving Go, but these aren't rare exceptions internally. What I see is that almost any data processing system with non-trivial logic can be improved significantly by applying modern machine learning. This might sound less than dramatic when put in those terms, but it's a radical change in how we build software. Instead of writing and maintaining intricate, layered tangles of logic, the developer has to become a teacher, a curator of training data and an analyst of results. This is very, very different than the programming I was taught in school, but what gets me most excited is that it should be far more accessible than traditional coding, once the tooling catches up. The essence of the process is providing a lot of examples of inputs, and what you expect for the outputs. This doesn't require the same technical skills as traditional programming, but it does need a deep knowledge of the problem domain. That means motivated users of the software will be able to play much more of a direct role in building it than has ever been possible. In essence, the users are writing their own user stories and feeding them into the machinery to build what they want.
Deep learning's eating software and software's eating the world. We just need a few waves of Chinese needle snakes to eat Deep learning. Then gorillas to eat the snakes. Finally when wintertime rolls around, the gorillas simply freeze to death.
Hi! I make Firefox Plug-ins. Check 'em out @ https://addons.mozilla.org/en-US/firefox/addon/youtube-mp3-podcaster/
Ya, I'm calling BS. Give us some concrete examples of how ML/AI/DL is doing anything other than burning CPU cycles on public clouds that drive up revenue for the cloud vendor.
Most software today and in the coming decades is designed and developed to support business processes or data flow and execution in scientific processes. These systems need a deterministic and foreseeable behavior. Yes, you may use "learning" classification mechanisms such as neural networks to support some tasks, but this is not changing how we develop software. Especially, developing software is usual a technical and social process, as you have to understand the demands and needs of users, which require interviews and discussions with users. You also need to communicate with UI designers to develop together with users and UI designers useful and easily to understand interfaces. And yes, you have to map all this onto technology.
Except for the fact the deep learning systems, may not be efficient enough. There are some tasks which it excels at, however some tasks there just isn't the volume or rewards for outcomes for the system to adapt fast enough to.
Lets say a flood control dam system. Which is is strictly coded if water goes above this line, open the dam and flood the down, because otherwise the damage is far greater than flooding the town. The reason why this is computer controlled is because it takes the faulty morals out of the picture. You wouldn't want to be the one to press the button and flood the town, even though it is for the greater good. A Deep Learning system, will want to take more variables into account, and will either decide that it is important to press the button before it hits the limit, as it would only create minor flooding problems, or too late figuring that it can wait a little longer creating a catastrophic failure.
Deep Learning often creates a lot of superstitious like behaviors (As shown in the funny walk of Googles Deep learning to walk video) Where it learned to do the task, but not at optimal way. This may just be a mater of time for the technology to get better, however it isn't a time to panic and toss out your CS Degree and get a job at your local Walmart as a greeter.
If something is so important that you feel the need to post it on the internet... It probably isn't that important.
ML is generally enabling scenarios that were just too tedious to actually do by developer hands. Sure there are specific scenarios where developers had done the best they can (and generally failed) with hopelessly unstructured data, but for the most part those problems were just left untouched as infeasible to do manually.
For the vast majority of software development, ML doesn't add anything. If you have no unstructured data or a way to impose structure, ML doesn't do anything over boring old programming. Even when you find yourself in one of the very chaotic, large, and diverse data sets where ML can in theory help you sort through, you have to first chew through enough data in training to get decent confidence. So you not only need a large data set, you also need to have a continued need after human assisted training has already done the work on a big chunk of that data. Even then you may be grasping for some intelligent way to apply ML techniques, because the kicker is you have to have some sort of real idea of what to do, even if you have a 'how to do it'.
Big Data has done this same song and dance. ML is now the purported answer to 'once collected and have tools to analyze, most orgs have no idea what to do with the data'. I suggest that the orgs will still have no idea what to do with the data, and ML won't move the needle much in the wider market because the root cause is just a general lack of thoughts on what to do with the data. This is the curse of hyped adoption, the vast majority of adopters will be disappointed because it doesn't magically solve.
XML is like violence. If it doesn't solve the problem, use more.
And by "deep learning" in most cases they mean "linear regression on cleaned-up data"
-------
1. Enjoy your job
2. Make lots of money
3. Work within the law
Choose any two.
Once upon a time, I did my doctorate in machine learning. The machines were less powerful, but the algorithms? Basically the same as they are today. Sorry, the stuff most widely in use is still the same back-propagating neural networks. The machines are just faster, so the networks can be bigger. That's it.
Neural networks can work really well on specific problem domains. The problem is: You have no idea what they are actually learning. The features that a network identifies within its layers are not really accessible to us. The problem lies, imho, in the total lack of domain knowledge. Since the network doesn't understand what the objects in those pictures are, they are doing a purely mechanical analysis of some (and who knows which) aspects of the pictures. They can learn some really weird things.
In a well-trained network, the results mostly coincide with our expectations. In a completely isolated domain, like chess or go, a network can be trained sufficiently to perform quite well. However, in open domains, they are fragile: we have no idea when they will break. Look at the video of the turtle being identified as a rifle (in the link above). Why does the identification jump seemingly at random? When will a cat will suddenly be guacamole? When will a pedestrian crossing the road will suddenly be just a pile of leaves? We have no idea, none.
It is certainly true that selecting and managing training data is a very different task from classic programming. However, it doesn't really take much domain knowledge. In most domains, gathering training data is tedious, not difficult. The hard part comes in figuring out how to make the best use of that data to train and test a network - and that requires a deep understanding of how the neural networks work (and how they don't work). Plus, frankly, a huge pile of trial and error, because there aren't many rules on how to best structure a net for any particular task.
Enjoy life! This is not a dress rehearsal.
...when you can input a photograph of an airplane and the Navier-Stokes equation, and get a flight simulator as output.
but they will take far too long to be useful for the microsecond resolution trading that is done today
A guy like Warren Buffett may think a year before doing a trade, and he's been pretty good at it. Not everything has to be done at microsecond level.
Most programming jobs involve connecting stuff together. Converting a database format to another, design a GUI around it, add the entry points to turn it into some kind of module, extract or integrate features, etc... Even machine learning typically involves gathering a bunch of data turn it into a form that's acceptable for the learning module and feeding the results to some other component.
I don't know how machine learning will help with all that stuff. An AI won't write a video game, it can help making mobs smarter, generating convincing maps or optimizing revenue. But in the end that's just a module connected to other modules, and programmers will be needed to put the round peg into the square hole.
It will make things a bit more high level, as always. But except for a bunch of PhDs, I don't expect major changes in the way people program.
I work on mostly CRUD and e-reporting applications. Generally an org wants these kinds of apps to be predictable and reliable, not "organic" (trial and error). I don't see organic learning as a viable way to program such in the future.
However, I can see AI being used to test the apps and find potential bugs in the source code, being that "suspicious pattern detection" is something it can do relatively well. It may also suggest code, schema, and UI refactorings. But such AI would be an adviser to programmers, not behind the wheel itself.
Clippy impression: "It looks like you have a lot of code pattern duplication between modules 7, 22, 43, and 51. Would you like help refactoring?"
Table-ized A.I.
For computational linguistics (translation, analysis, etc), machine learning is not a net gain. What ML proponents forget to factor in is the vast time spent on gathering and hand-annotating large quantities of text (gold corpora).
Even worse, for many many languages, these gold corpora simply do not exist and there are no plans on making them, or they are too small to be used for ML.
And even when the gold corpora do exist, models trained on them become tightly coupled with the data. They become domain specific. In order to escape domains, you need an order of magnitude more data.
Instead, one can make a domain-independent rule-based system in a fraction of the total time spent on machine-learning models. But rule-based has become this weird anathema - people will even write papers that use rule-based methods, while hiding it behind machine-learning terms.
I'm sure this also holds for other fields.
In my line of work I use a lot of mathematical optimisation. As Stephen Boyd says in his course, everybody working in optimisation has at some point this epiphany: "everything is an optimisation problem". And this is true. However to make it work you need to be very good at mathematical modelling, you need to know your methods, and most of the time the problem is unsolvable anyway by the classic methods.
In this instance maybe a lot of programming can be modelled by some deep NN. However you have to come up with a relevant architecture for your problem, you need to train it, and you need to evaluate it. It may save you time to do so, but if you need so solve something like FizzBuzz, that may not be the best way.