[infurl]

https://distill.pub/2018/building-blocks/

This is a really great article about how to make sense of what is actually going on inside a neural network.

"With the growing success of neural networks, there is a corresponding need to be able to explain their decisions — including building confidence about how they will behave in the real-world, detecting model bias, and for scientific curiosity. In order to do so, we need to both construct deep abstractions and reify (or instantiate) them in rich interfaces."

[LOCKSUIT]

http://playground.tensorflow.org

All the newcomers only need to know a few things to get the idea of ANNs. It's a shared-feature hierarchy that re-uses features like curves and noses. First it detects lines/curve types, and depending on which are more important it has learned or how many are present, it will activate maybe a nose and eyes in the next layer, then finally a face in the next layer. Same for anything you recognize in your day. This is Discrimination/classification. Generation is the opposite way - top-down. Look up GANs. They are incredibly useful. You select say a bedroom and it creates as high of resolution images you desire that are new and as asked for even by sentence input and can even predict next frames to generate video.

[infurl]

Yes Locksuit that is as good an explanation of how they work as any, however it is missing the point of this particular article.

The new research is concerned with how to identify (reify or name) the elements that the neural network is using. We understand how neural networks are doing what they do, but to trust them we need to understand what they are doing in each instance as well. There should be a way to prove that it is correct in a language that we understand.

There is another important aspect to this as well. Neural networks require vast amounts of training data and computation, putting their creation out of the reach of people with limited resources. Maybe this form of analysis would allow neural networks to be programmed rather than taught, and maybe even achieve something that animals can do easily, which is learn from a single example.

[LOCKSUIT]

Don't we alrady have a way to see that it is learning+what it's learning during training? We have it Generate samples during training.

Also what about this?:


Also does the nodes in the link below show images of what each neuron is seeing? Look closely at the nodes. It shows you what they have learned.
http://playground.tensorflow.org

[infurl]

You're right about already being able to see what's going on inside the neural network, but seeing and understanding are different things. The goal of the work in the original article is to assign meaning and condense everything down to a scale that is comprehensible by mere humans. For example they isolate the internal elements that allow a neural network to detect floppy ears and all their variations. The techniques that you describe are combined with a variety of methods to accomplish that goal.

Edit: keghn posted a good video that accompanies the article over here http://aidreams.co.uk/forum/index.php?topic=12802.msg51110#msg51110