Slashdot Mirror
Deep Learning Is Eating Software (petewarden.com)
Pete Warden, engineer and CTO of Jetpac, shares his view on how deep learning is already starting to change some of the programming is done. From a blog post, shared by a reader last week: The pattern is that there's an existing software project doing data processing using explicit programming logic, and the team charged with maintaining it find they can replace it with a deep-learning-based solution. I can only point to examples within Alphabet that we've made public, like upgrading search ranking, data center energy usage, language translation, and solving Go, but these aren't rare exceptions internally. What I see is that almost any data processing system with non-trivial logic can be improved significantly by applying modern machine learning. This might sound less than dramatic when put in those terms, but it's a radical change in how we build software. Instead of writing and maintaining intricate, layered tangles of logic, the developer has to become a teacher, a curator of training data and an analyst of results. This is very, very different than the programming I was taught in school, but what gets me most excited is that it should be far more accessible than traditional coding, once the tooling catches up. The essence of the process is providing a lot of examples of inputs, and what you expect for the outputs. This doesn't require the same technical skills as traditional programming, but it does need a deep knowledge of the problem domain. That means motivated users of the software will be able to play much more of a direct role in building it than has ever been possible. In essence, the users are writing their own user stories and feeding them into the machinery to build what they want.
ML is generally enabling scenarios that were just too tedious to actually do by developer hands. Sure there are specific scenarios where developers had done the best they can (and generally failed) with hopelessly unstructured data, but for the most part those problems were just left untouched as infeasible to do manually.
For the vast majority of software development, ML doesn't add anything. If you have no unstructured data or a way to impose structure, ML doesn't do anything over boring old programming. Even when you find yourself in one of the very chaotic, large, and diverse data sets where ML can in theory help you sort through, you have to first chew through enough data in training to get decent confidence. So you not only need a large data set, you also need to have a continued need after human assisted training has already done the work on a big chunk of that data. Even then you may be grasping for some intelligent way to apply ML techniques, because the kicker is you have to have some sort of real idea of what to do, even if you have a 'how to do it'.
Big Data has done this same song and dance. ML is now the purported answer to 'once collected and have tools to analyze, most orgs have no idea what to do with the data'. I suggest that the orgs will still have no idea what to do with the data, and ML won't move the needle much in the wider market because the root cause is just a general lack of thoughts on what to do with the data. This is the curse of hyped adoption, the vast majority of adopters will be disappointed because it doesn't magically solve.
XML is like violence. If it doesn't solve the problem, use more.
...when you can input a photograph of an airplane and the Navier-Stokes equation, and get a flight simulator as output.
Most programming jobs involve connecting stuff together. Converting a database format to another, design a GUI around it, add the entry points to turn it into some kind of module, extract or integrate features, etc... Even machine learning typically involves gathering a bunch of data turn it into a form that's acceptable for the learning module and feeding the results to some other component.
I don't know how machine learning will help with all that stuff. An AI won't write a video game, it can help making mobs smarter, generating convincing maps or optimizing revenue. But in the end that's just a module connected to other modules, and programmers will be needed to put the round peg into the square hole.
It will make things a bit more high level, as always. But except for a bunch of PhDs, I don't expect major changes in the way people program.
Essentially, ML can replace (parts of) systems that rely on heuristics. Anything with fixed rules, no matter how complicated the rule set, will not benefit. Why train a ML system when you can get 100% deterministic answers?
For computational linguistics (translation, analysis, etc), machine learning is not a net gain. What ML proponents forget to factor in is the vast time spent on gathering and hand-annotating large quantities of text (gold corpora).
Even worse, for many many languages, these gold corpora simply do not exist and there are no plans on making them, or they are too small to be used for ML.
And even when the gold corpora do exist, models trained on them become tightly coupled with the data. They become domain specific. In order to escape domains, you need an order of magnitude more data.
Instead, one can make a domain-independent rule-based system in a fraction of the total time spent on machine-learning models. But rule-based has become this weird anathema - people will even write papers that use rule-based methods, while hiding it behind machine-learning terms.
I'm sure this also holds for other fields.