[Korrelan]

@Kei10

I presume you mean the sigmoid based neural net not the classic perceptron.

A perceptron can only output 1 or 0, a sigmoid neuron can output all values between 0 and 1, ie 0.3456 etc. It’s the sigmoid that’s usually used in image recognition etc.

In neural network, how do they store memory?

Think of a sigmoid neuron as a math function (Tan, Cos) it has no memory. The connections and weights act as a mathematical filter based on the sigmoid function. Though I suppose you could imagine the trained weights as a kind of memory filter.

A normal memory search involves scanning a list for a result. The sigmoid neuron will always give the same output for the inputs because of the trained weights and bias.  So rather than scanning a list in memory, you are feeding the list through the neuron which will fire when it detects the correct input.

Wouldn't that training using data Y will overwrite the weights to output W?

If you were training just one neuron yes, but useful NN have the hidden layer were the multiple inputs can be split and blended through backprop to the correct weights required to recognise the two distinct inputs.

This is a very good explanation of the sigmoid neural network architecture. It’s a bit long but contains some very cool interactive graphs to help the explanations.

http://neuralnetworksanddeeplearning.com/chap1.html

I think you will find chapter 4 very interesting.

@BF33

I’m pleased you feel you have learned enough about CNN to understand them.

 

[kei10]

@korrelan
I see, so that's what it is! This is really interesting!
Thank you for your response!

Now I'm hyped to code some neural network stuff. I wonder what's the simplest project I can get started... Hmm...

 

[LOCKSUIT]

Wait wait wait...when input enters the CNN in the brain, does the image I'm looking at parallel-ly pass ALL stored visual hashes? Ex. hash of mom's face, hash of dog seen 8 years ago, hash of a violin.

[keghn]

 On the output side of 500 x 500 pixel side of The CNN only one pixel goes from zero to one to indicate a detection of
a violin. You can make it fancy by make it out put a few bytes on detection. This is encoding.

[LOCKSUIT]

That doesn't answer the question.....................

To be more clear > After all your life 84 y/o you have billions of saved images, as hashes, that have been compressed by filters so each pixel is a sum of a filter. Then mr. wiggles comes in, he's a new comer to town, and he's sent straight to Memory. Is he split into billions of clones (torture) and then all his clones (and him) are brought to the next room - where *they *all pass *all *stored images *parallel-ly?

[keghn]

 A cnn is a detector network. It store nothing. it only activates when pointed at the target that it was trained for and
turns off when target is out of range or out of the picture. A computer can record the time activation happend and
along with the hash code. A sequence of hash code can be recorded. And the hash number can be used over again like world in a sentence.

[LOCKSUIT]

Drew Picture:
http://advancessss.deviantart.com/art/Convolution-NN-652106540?ga_submit_new=10%253A1482162381

Is this basically how it works? (look at my picture)
- Image I'm looking at enters (of my brother).
- It is convolved down.
- Cloned (how ever many times depends on stored hash amount ex. 98 billion times).
- Each clone passes each stored hash at the same time.
- The features spark and make the Output layer fire only 1 selected memory image.
- And therefore only a certain saved actions are done (says hello I love you my brother).

[Korrelan]

Hope you guys don’t mind me jumping in… I've took the same due diligence and learned from your example lock.

All your questions and answered in one single graphic.  I even took the time to convert/ encode it into a style you obviously enjoy/ understand… see how considerate I am?



See all clear now… you should have no further queries or misunderstandings.

 

[Freddy]

I like how you used a stick man to represent the human condition.

[keghn]

 In NN that detects. The memory that is used inside holds a value that is a amplification value. The amount to amplify
or de-amplify a signal coming off a pixel and going into a neuron, of the detector NN.  This causes a image of your
brother not to be stored in one location. All or a lot of the million neurons will hold a small piece of your brothers
image. like a hologram. NN detector is like a special lens only the light of your brother will make it through the NN.
A NN can be trained to do your brother and 1000's other people or objects. Anything else that was not trained
into the NN will not activate it.

[LOCKSUIT]

So korrelan you're saying the input image is convolved into a small hash and then goes STRAIGHT to the match and then speaks the linked actions? Without figuring out if the input image (snake) hits more features in the snake image than ex. a grandpa image? IF input actually only passes a few filters ex. curve blob and etc then how can that do a good job you'd need manyyyyyy, that's why I thought each image makes its own filter !

[Korrelan]

Do crabs think fish can fly?

 

[LOCKSUIT]

I studied up more on CNNs, and have a newer question to ask.

If you have 1 convo, 1 layer, and 1 million possible outputs (FULLY intact stored images at the end of the network), then how many filters? Does there have to be as many filters as there are output selections? Or would 100 filters work for only choosing 1 of the 1,000,000 stored images?

[Korrelan]

Ok… one last time… I’ll try to use short paragraphs.

If you have 1 convo, 1 layer, and 1 million possible outputs (FULLY intact stored images at the end of the network), then how many filters?

It depends on what kind of convolution has been applied to the input image and the ‘feature maps’ and the resolution of the input etc.

In the example image the ‘convolution filter’ applied to both the feature maps (A) and the input image (B) is a contrast filter.  This converts the normal RBG colour pallet into black and white; emphasising the contrast gradients in the input image (B).

This simple example shows only eight ‘feature maps’ (A). The maps are scanned each/ individually against the input image (B).  An example of the path the ‘eye’ (highlighted red) feature map is applied to the input image (B) is shown by the red arrow. 

When a ‘match’ is found in (B) the position is noted and the feature map is drawn at the same x,y cords in (C). 

So…the image (C) is made up of ‘where’ the eight ‘feature maps’ where ‘matched’ in the input (B). Image (C) is constructed from the eight feature maps.

Most systems record the locations where the feature maps (A) where ‘matched’ in the input image (B) as a set of vector coordinates (a list).

The CNN never has to store whole images; using just eight feature maps (more is better) you can describe thousands of objects. You only need to store the feature map numbers and their relative positions to each other.

So for the face (C) only the feature map numbers (A 1 to 8 ) and their relative x,y coordinates to each other has to be saved to enable this face to be recognised in other input images.

This greatly compresses the required memory required for the CNN and vastly speeds up processing.

This is a very simple CNN example.

 

[LOCKSUIT]

You didn't answer the question...

My system HAS to store FULL images (at the end [END]) so that it can remember them AND see em in dreams.

Everytime a image enters the brain it has to select 1 of the already stored images.

The way it knows to select which one is by the activation energies that it has to play with ex. softmax 1.0000000.

The question is simple. - If there is currently 10 billion images stored in memory (and yes the features not just the full guys at the end) - will there be 10 billion features (each made when a image enters) OR will there only be "veteran" features EX. 10,000 in this brain?