DeepMind Open Sources 'Sonnet' Library For Easier Creation Of Neural Networks (fossbytes.com)

"We are very excited about contributions from the community," announced Alphabet's DeepMind, open sourcing a new library to make it easier to build complex TensorFlow neural networks. An anonymous reader writes: "DeepMind foresees Sonnet to be used by the community as a research propellant," reports FossBytes. "Also, it would allow easy sharing of other models created by DeepMind with the community." Sonnet uses an object-oriented approach, a recent blog post explained, pointing to more details on GitHub. "The main principle of 'Sonnet' is to first construct Python objects which represent some part of a neural network, and then separately connect these objects into the TensorFlow computation graph."

DeepMind sees this as part of their broader commitment to open source AI research. "In recent months we've also open-sourced our flagship platform DeepMind Lab, and are currently working with Blizzard to develop an open source API that supports AI research in StarCraft II."

Comments

Re:Got one thing right...

[Anonymous Coward]

I'm pretty avid about keeping in the loop with Machine Learning and I don't know TensorFlow well enough to understand the benefits of this.

I'll say that TensorFlow by itself without TFlearn or Keras is pretty obtuse, even for someone who understands the theory fairly well. I can fluently use words like mini-batch normalization, max pooling, drop out, ReLU, and convolution kernel in a conversation, but even the basic MLP TensorFlow MNIST example isn't exactly easy to understand (and yes, I know the linear algebra and multivariable calculus involved in both multiplying the weight matrices, transfer funtions, and the backpropagation partial derivatives).

I would compare learning to TensorFlow (equipped with an understanding of theory, Math, and Python) to learning to build Android applications (with a solid background in Java application development). There's a huge learning curve associated with the tools that will not be immediately transparent even for those that you would think should be able to hit the ground running.

Frankly, I can look at Torch code and understand exactly what is going on without digging through the Torch documentation (TensorFlow is at least better documented than Torch).

I think Sonnet is supposed to take some of the "sting" out of this learning curve, but my first look at the code examples left me underwhelmed in terms of what gains were being made here.

Feeding the application non-standard datasets is still the most annoying part about all of this. I know it can be difficult to accommodate every edge-case, but IMHO, it is WAY too hard to feed TensorFlow a .csv file, or a labeled folder of images (something NVIDIA Digits does really well).

MATLAB is actually pretty good in this regard; however, their Deep Learning Toolbox is years behind the bleeding edge with pitiful GPU support.

Ditto. Much like assembly to C to C#

[raymorris]

Agreed. It seems we've gotten over a hump going from machine learning etc being a research area to where it's now usable for real-world applications. However, it's currently usable only for specialists. A major breakthrough will be when it's abstracted and packaged to be usable by ordinary developers.

Once upon a time, to write programs for a particular CPU, you had to know about the details of that particular CPU hardware, its registers and all that, and write in the appropriate asssembly language. Later, you could write software in C - not only did you not need to understand the details of the CPU in order to write software that uses the CPU, but the same C code will run on different CPU architectures. This is because the people who do know the details of the CPU hardware wrapped that in an abstraction layer. Similar for GPUs. Once upon a time, graphics programming required fully understanding the details of the particular graphics hardware in use. Now, general developers can work with hardware accelerated graphics as ordinary objects C#, courtesy of DirectX.

It feels like machine learning is now mostly at the "write in assembly" stage, you have to thoroughly understand how the thing works in order to use it. There are some developer-friendly libraries, like Intel's machine vision library. One day we'll have a full suite of developer-friendly libraries. We'll be able to call Pattern Finder and Pattern Matcher and have them find and match patterns, without knowing or caring HOW it matches patterns. Hopefully it'll be at least as easy as how Perl programmers use regular expressions all day, without having a clue how the regex engine works internally.