DeepMind Open Sources 'Sonnet' Library For Easier Creation Of Neural Networks (fossbytes.com)

"We are very excited about contributions from the community," announced Alphabet's DeepMind, open sourcing a new library to make it easier to build complex TensorFlow neural networks. An anonymous reader writes: "DeepMind foresees Sonnet to be used by the community as a research propellant," reports FossBytes. "Also, it would allow easy sharing of other models created by DeepMind with the community." Sonnet uses an object-oriented approach, a recent blog post explained, pointing to more details on GitHub. "The main principle of 'Sonnet' is to first construct Python objects which represent some part of a neural network, and then separately connect these objects into the TensorFlow computation graph."

DeepMind sees this as part of their broader commitment to open source AI research. "In recent months we've also open-sourced our flagship platform DeepMind Lab, and are currently working with Blizzard to develop an open source API that supports AI research in StarCraft II."

Comments

Easier for data scientists

[Anonymous Coward]

After recently diving into machine learning and playing with the available libraries out there. IMO what is really needed is libraries that are developer friendly, not data scientist friendly. That's when we will see a lot more developers starting to apply this to more real world problems.

I know enough to know that I don't need to know that much about the algorithms involved to tweak them towards an end goal. I just need to know ok this knob moves it this way, this algorithm is best for this type of problem, etc..

I don't want to learn scientific notation, I want a library for developers by developers not for data scientists by data scientists.

I started with a specific problem I wanted to solve, and pretty much everything I could find out there was student projects doing the same things over and over. Almost nothing about how someone applied this to a real world problem.

Don't get me wrong I'm not knocking on the work that's been done. More like frustrated because right now i have to learn far more then I really should have just to solve some practical problem using machine learning.

Re:Easier for data scientists

[Anonymous Coward]

Look up Sirajology on youtube.

Re:Easier for data scientists

[Anonymous Coward]

I liken an AI API to the state before printer driver development 35 years ago. Earlier in the 70s, each printer had its own specialized API. What we had back then were software houses who needed to apply their code to many different printers and so the idea of drivers was adopted to sit between the application software and hardware. AI of today can't progress until common functions of all the AI platforms can be expressed in a way that saves the independent developer from the dreaded lurgi of 'lock-in' and that's a chicken and egg situation.

Hardware

[Anonymous Coward]

If google just could start selling their TPU processors in a nice and bundled format, maybe as PCI express card or something... I don't want to buy nvidia shit and have to endure its closed drivers on linux, nor do I want to buy from google's cloud 'offer'.

Got one thing right...

[__aaclcg7560]

Works with Python. Cool! What does it do?

Re:Got one thing right...

[Anonymous Coward]

I'm pretty avid about keeping in the loop with Machine Learning and I don't know TensorFlow well enough to understand the benefits of this.

I'll say that TensorFlow by itself without TFlearn or Keras is pretty obtuse, even for someone who understands the theory fairly well. I can fluently use words like mini-batch normalization, max pooling, drop out, ReLU, and convolution kernel in a conversation, but even the basic MLP TensorFlow MNIST example isn't exactly easy to understand (and yes, I know the linear algebra and multivariable calculus involved in both multiplying the weight matrices, transfer funtions, and the backpropagation partial derivatives).

I would compare learning to TensorFlow (equipped with an understanding of theory, Math, and Python) to learning to build Android applications (with a solid background in Java application development). There's a huge learning curve associated with the tools that will not be immediately transparent even for those that you would think should be able to hit the ground running.

Frankly, I can look at Torch code and understand exactly what is going on without digging through the Torch documentation (TensorFlow is at least better documented than Torch).

I think Sonnet is supposed to take some of the "sting" out of this learning curve, but my first look at the code examples left me underwhelmed in terms of what gains were being made here.

Feeding the application non-standard datasets is still the most annoying part about all of this. I know it can be difficult to accommodate every edge-case, but IMHO, it is WAY too hard to feed TensorFlow a .csv file, or a labeled folder of images (something NVIDIA Digits does really well).

MATLAB is actually pretty good in this regard; however, their Deep Learning Toolbox is years behind the bleeding edge with pitiful GPU support.

Re:Got one thing right...

[Anonymous Coward]

Works with Python. Cool! What does it do?

If I'm understanding this correctly, this is automating the processing of sequences of vectors. I'm understanding it in the context of Recurrent Neural Networks. Sonnet seems like Google's attempt to introduce modularity to the process of machine learning so that you could, for example, execute optimization over groups of neural networks.

Error During Build

[superid]

I'd like to try this but the build is failing on a newly updated ubuntu system.

bazel build --config=opt :install

C++ compilation of rule '//sonnet/cc/kernels:resampler_op' failed

Has anyone had success following the build instructions here: https://bazel.build/versions/m...

Re:Error During Build

[Anonymous Coward]

Same problem here...

Ditto. Much like assembly to C to C#

[raymorris]

Agreed. It seems we've gotten over a hump going from machine learning etc being a research area to where it's now usable for real-world applications. However, it's currently usable only for specialists. A major breakthrough will be when it's abstracted and packaged to be usable by ordinary developers.

Once upon a time, to write programs for a particular CPU, you had to know about the details of that particular CPU hardware, its registers and all that, and write in the appropriate asssembly language. Later, you could write software in C - not only did you not need to understand the details of the CPU in order to write software that uses the CPU, but the same C code will run on different CPU architectures. This is because the people who do know the details of the CPU hardware wrapped that in an abstraction layer. Similar for GPUs. Once upon a time, graphics programming required fully understanding the details of the particular graphics hardware in use. Now, general developers can work with hardware accelerated graphics as ordinary objects C#, courtesy of DirectX.

It feels like machine learning is now mostly at the "write in assembly" stage, you have to thoroughly understand how the thing works in order to use it. There are some developer-friendly libraries, like Intel's machine vision library. One day we'll have a full suite of developer-friendly libraries. We'll be able to call Pattern Finder and Pattern Matcher and have them find and match patterns, without knowing or caring HOW it matches patterns. Hopefully it'll be at least as easy as how Perl programmers use regular expressions all day, without having a clue how the regex engine works internally.

Re:Ditto. Much like assembly to C to C#

[Anonymous Coward]

Except I wouldn't get your hopes up. The current machine learning algorithms haven't changed in decades. It's just now the speed in hardware is faster to make it feasible. At the end of the day "Machine Learning" requires knowledge of statistics and the data being used.

Re:Ditto. Much like assembly to C to C#

[Anonymous Coward]

Here are values that give YES, here are values that give NO. Use evolution algorithm to figure out best way to work with this kind of data.

Shouldn't be impossible to write.

Re:Ditto. Much like assembly to C to C#

[Anonymous Coward]

Later, you could write software in C - not only did you not need to understand the details of the CPU in order to write software that uses the CPU, but the same C code will run on different CPU architectures

At least that's the theory. In practice, anything more complex than writing text to the screen required system-specific programming - hence the long term separation between Windows and Unix programs.

Re: Ditto. Much like assembly to C to C#

[Anonymous Coward]

This discussion us relevant to the last slashdot article on how Google does open source and why. My comment is the same. They can dictate the way you apply machine learning I.e. the way they want you to use it and what they want you to use it for - in what context.

Re:Ditto. Much like assembly to C to C#

[Blaskowicz]

And soon, there won't be programmers anymore because computers will program themselves :)
You only have to write the program's specification (a square mile of UML diagrams will do).

Re:Ditto. Much like assembly to C to C#

[Blaskowicz]

Choosing the values might be hard?
Not only there's the problems of discarding, clamping, correcting data (if you remember "hide the decline", maybe that's stuff you will have to do) but I wonder if you'll end up doing something silly. E.g., your system ends up increasing the production of men's underwear when the number of pirate attacks is low. Then next year a outlying bout of 1000 pirate attacks happens, your program falls apart but before the underwear company notices they shouldn't blindly trust what comes out of the computer, they will miss a ton of sales and their underwear competitors will mop the floor with them.