[minomic]

I am programming on MATLAB and want to use RBMs with real-valued input, like greyscale images, so I tried to follow what Hinton said in this article: http://www.cs.toronto.edu/~fritz/absps/esann-deep-final.pdf

The images have integer values in [0, 255] and are stored in a matrix D which is [numImages x numPixel]. So I started preprocessing the data:

- scaled the entire dataset so that all the values are in [0, 1] with

D = D / 255;

- brought every pixel to have zero mean across all images, so I subtracted from every column of the matrix its mean value with

imgMean = mean(D); % row vector
D = D - repmat(imgMean, rows, 1);

- divided the entire dataset by its standard deviation, so that every pixel has unit variance, with

D = D / std(D(:));

But when I try to plot the images, the result is clearly very dark, since many values become negative and are clipped to zero.

Is this ok or did I make any mistake with the preprocessing?

[Ultron]

But when I try to plot the images, the result is clearly very dark, since many values become negative and are clipped to zero.

Is this ok or did I make any mistake with the preprocessing?

Does the resulting image appear as an inverted grey scale version of the original? If so, than you can simple invert / swap the values with their opposites (1 with 0 and vice-versa). However, if the image is not inverted, yet simply a black and white (2-color) version, then it is a bit more complicated.


I cannot give you an exact solution, but I can advise you to read some more on image processing and value conversions (graphical), unless you already haven't done an extensive study. Also, the GIMP project is an open-source image manipulation program. It contains functions that convert images to grey-scale and many different parameters. You might find some of the code useful, so check out:


http://www.gimp.org/ and http://www.gimp.org/source/

[minomic]

No, unfortunately it does not appear as an inverted version of the original. It is very dark since I have many negative values that are zeroed.

Actually, I would like to know if this way of preprocessing data is correct or if I should perform another kind of shifting and scaling.

[ranch vermin]

how big is the network.  the smaller it is the less it will be able to store.

Im no expert on these,  but ive done a little myself,    I could get a hopfield running, but im not fully up to making a restricted boltzmann machine,  as they are different in a way im not sure about.

I actually made something like a restricted boltzmann, but it was much simplified.

All I did was increase the connection toward the data and decrease its reconstruction of the data, not worrying about probabilities or energy at all,    You actually can completely screw it up and it should still work (repeating back the data anyway),   just not work with a fully complete state recovery.

Maybe it could be your initial randomization of your weights?

[minomic]

The images are 60x54, so 3240. The network is composed of four layers with size 4000 - 2000 - 1000 - 500, and the symmetrically. So the result is
3240 - 4000 - 2000 - 1000 - 500 - 1000 - 2000 - 4000 - 3240

As far as randomization of the weights is concerned, I followed what Hinton did: 0.1*rand, so that we are sure the values are small.

Actually all this mess with pre-processing comes in because I read (in Hinton's work) that logistic units are ok with data like MNIST handwritten numbers (since they are almost binary) but they are not a good choice to handle images, since there is much more variation.

So the idea was changing the first RBM from binary-binary to linear-binary, and then to go on with the binary-binary layers.

[ranch vermin]

If I went to try and make a boltzmann again id do this  (but it goes to show how little I know about them)

This isnt an explanation its more me trying to learn it the same time as you.

Doing it logisticly->

a) clamp the data to the visible units,  and then go to energy minima on the hidden states over a set of 10 or so steps, just done parallel , who cares about oscillation...

b) with no data clamped, i then go to energy minima for the whole network, the same way, just letting it run wherever it wants.

c) then I train it-> i then increase the weights connected to cells that are on in the visible clamped energy minima, and minus when it isnt clamped.

And that would be if it worked an unoptimized way to do it, and its incremental, would have to do its deepness serially,and its really forgetful.


I guess the advantage would be you can keep adding cells as well as synapses to get over spurious patterns/states/keys.

If you got video to play back out of it randomly going backwards and forwards, your going good.