[LOCKSUIT]

Hi everyone!

What started me off onto AI was a video of stick creatures getting better at learning to gallop faster.

I learned fast how it's all working and was quite amazed, I figured it out without really knowing how Q-Learning/etc. worked.

That left room for two different working theories, or, understandings.

I'm going to show you the video below.

First I'll say my explanation of the AI's main parts. Then I want to hear your explanation of how it's working. And compare our explanations.

My explanation:
1. It generates and saves motor actions in memory.
2. Does the motor actions.
3. Sensors get senses.
4. Send them to the rewards.
5. Saves the senses in memory linked to actions just tried and are labeled + or - and ranked by the setup rewards.
6. Senses then match memory to the highest rank which must be at least similar to the sense matching memory.
7. The chosen actions+senses go back up to rewards and down to memory front and are actions done here.
8. So it's doing the best actions when it gets a similar sense to the sense that rewarded it or a sense linked to that by matching memory.

[Freddy]

Yes I like that video too, it's surfaced here before. I never looked into how it's working. I need another lifetime !

[Art]

So I guess we could always go Bach to basics...

[LOCKSUIT]

Art, what are you saying? Why am I not getting answers from yous? I thought experts are here???

Like, I would like you to not just give your explanation, but also if mine is correct, if not very clear what the stick creatures are doing.

Why must this be so hard.....

[Korrelan]

Hi BF33

Your explanation of how this simple system works is wrong I'm afraid.

It’s not as simple as saving (+/-) values to sensory inputs.  There is no reward system here either.

It’s based on a genetic algorithm.  The attributes of the ‘muscles’ and springs are based on a simple set of Genes.  At the end of the simulation run the bot that wins, or comes closest to the set required critea is used as a template (Genes) to generate the next set.  It’s basically survival of the fittest or selective breeding.

The simulation uses a classical neural net with 52 inputs.  Neural nets do a lot more than simply save (+/-) values against set/ random actions.  It’s the complex overall weights from the inputs that dictate whether a neuron fires.  There would be and input/ hidden and output layers of several neurons. The neural net is able to take temporal timing into its calculations, something a simple list cannot do.  Neural nets also generate gating functions, NAND, OR, AND, etc… again a list cannot do this.

As a thought experiment let’s try to simulate one classical neuron with a list.  The list would comprise of entries for the inputs, let’s say the neuron has only five (usually many more) and only uses integer values…

1,2,3,4,5    +  1
3,3,2,1,6 +  2
4,4,4,2,3 +  2
5,4,3,1,2 -   4
8,7,6,1,2 -  1

You would need one list for each neuron.  The list would have millions of entries to cover all the possible permutations of inputs/ outputs.  How would the system know the + range of correct entries at the top (assuming correct entries are shuffled to the top, to simulate OR gates, etc). How does the system know where to send the output?  You need a connectome of some description for the linking of lists.

If you were to just try and use one list for the whole system, even changing one input or output value of one sensory input would require the whole list entry to be duplicated except for the one change.

The storage and processing power required to run even a simple simulation using this method (which wouldn't work) would be astronomical (impossible).

This simulation has 52 inputs (52 factoral)… so that’s 52 x 51 x 50 x 49… etc… that comes to 8.06e+67… or…

80,658,175,170,943,878,571,660,636,856,403,766,975,289,505,440,883,277,824,000,000,000,000.

List entries… And that’s without the outputs…. Hmmmm… I think not.

Edit: Thats assuming that each sensor value 1-52 only appears once
per list entry (just to keep it simple)

@Art... Bach to basics... hehe

[ranch vermin]

nah what u said was spot on,  now i dare you to actually implement and engineer it.   thats the hard thing,  the theory is childlike.


What kind of algorythms go into making one these machines?

do u think its easy to code one of these - one part of one way of making the reward based motor possible?   its a feedback model, of a moving image plane (its his eye, or perspective model.),   they break up after 2 seconds, or even less, and u only get a glimpse of the computer 'controlling itself'  or 'machine dreaming'



There is a solution to the problem,  but im keeping it a secret, just in case you are the enemy conspiring against me. And im not even going to tell you why you might want to implement this.  Or even how to do it.

Im just showing because you said their werent experts (or in other words, actually implementing.) hanging around here,  like I and Korrellan.


One thing i will tell you from my experience,  it doesnt actually matter if your robot actually works in the end, or you end up getting there,  because theres so many algorythmic gifts along the way its a worthwhile path to take.

[LOCKSUIT]

Ok ok, first I wanna say I did create my AI in a robot and it works just as good, see it below at 5:34. This totally tips over what you just said I couldn't do I did..... :O


K, now, you're saying the stick creatures are trying random actions right? That is what I said in my first post. Then you say when one gets the farthest right right? I said in my first post this is the "reward" trigger, (which must be switched to the rewards we love). And actions are tweaked when done.

Next, you say it has only 1 neural net for a arm/etc. and improves it, but mine also has the whole menu of motors and can adjust those around by tweaking the used motor actions just a bit. It saves the whole menu tried each time, and ones that are ranked&labeled are used and can be added/extracted certain motor actions ex. walk only and copy someones tap-ipad actions and so the matched tap-ipad action is sent to memory front and gets combined into the menu of motors.

I noticed you said temporal timing, this is not only random motor actions generated but also their length of them is randomized and done at specific times? The action saved and done should be what it is, if it has a stop moment, it has a stop moment, like grab for 3 secs>stop for 1 sec>pull for 5 secs. As for when to be done, read below:

Are you saying the stick creatures actions are fired on their own!!!??? All of a humans actions we have are ONLY fired when the sense cue is sensed, ex. see a handle or glass or hallway etc, because it matches that first sense from the first time which's linked to the actions to do(highest rank is picked as long as sense is similar to matching sense).

Also note that if you wanted to say actions its anticipating are done next, say that, because with mine I have how it links senses&actions just gotten/done to the next and last sets and can be linked to any new ones by being sent to mem front from anywhere it was.

((((Now I looked it up in this link and yes I have how mine does these truly like us https://www.willamette.edu/~gorr/classes/cs449/Temporal/tappedDelay.htm))))

Lastly I have mine does OR NAND AND etc, it links one of those sense cues I said and if it's false it can link to check another one. I have why you type a same word twice or backwords or miss some or say crappier other ones if write fast, you link neutral senses to real +/- rewards triggers and you rid the words already done and left to say the last, and say the first one wrong since in-short a mem match wasn't able to be done, oviously saying "I I want" twice is it had not being linked to a semi-trig therefore and done again and then is~ Mine works splendily with this is what I'm saying I know exactly how this all works.

[ranch vermin]

look at this -> https://www.pagiamtzis.com/cam/camintro/

neural networks are TOO SLOW, these are the quickest neural networks replacements.   but in software you have to do it in this way ->


]


if u dont understand that, ill tell you how the proper way -  just think, you have a 32 bit computer,  means you have 4 gig,  it means that you have every permutation of 32 bits.

this gives you 32 bit sensor. (4x8 binary eye) pure physical space,  but virtual space gives you 1000 bit sensor or more.  (32x32 binary eye)  but goes abit slower.

[Art]

Sorry Flower, but and Outline of the Human Brain is one thing. Your choice in music basically caused me to mute the sound and skip through a lot of the video.

[LOCKSUIT]

Ranch, are you saying I must search through all possible actions/set of actions tried? I don't need to, it can try the same one again~ I have a special way of locating a match too - by same spike strength pathway it fires right to it!

Nextly why did you not reply to each of my paragraphs like I did???
THE STICK CREATURES ARE GUESSING>FARTHEST IS THE SENSE REWARD TO RANK GUESS>BEST GUESS IS DONE+TWEAKED WHEN SEE NEXT TIME.

Remember my explanation at top I wrote, the steps, it is simpler said than what you said, it is WHAT it is DOING.

Sensors
Rewards
Memory
Guess/tweak
Motors

[Korrelan]

Hi again BF33…

Mine works splendily with this is what I'm saying I know exactly how this all works.

You use the phrase ‘I have how’… this works, an awful lot.  Now if you’re just telling people that you have discovered AGI and want a pat on the back then cool… Well done.

But… if you are trying to get other people to understand the workings of your theory… then we need to see the logic/ workings/ algorithms… at least something we understand in main stream terms.  Engineers produce diagrams/ blue prints/ circuit diagrams, etc… using a commonly/ mutually understood language/ syntax.

Regarding your last post…

The stick figures aren't guessing. They are behaving in a manner governed by their Genes. There is no reward at the ‘end of the race’… the two or more best performers are spliced to create a new neural net that should… but not always perform better at the given task.

Sensors, Rewards, Memory, Guess/tweak, Motors

This would have to happen after every single frame of movement, how would you reward or guess/ tweak when you can’t calculate how well the bot is performing until it’s finished it’s task?  A bot mign’t perform 1000’s of actions to achive a goal, how do you tell it which where correct/ incorrect at the end of the task?

Also… In real world tasks sensors would come before motors, it’s a bit late to not reward the bot after it’s just crushed someone.

Let’s just take one aspect of your system.  A single eye. 

Forget all the rest of the system. Just explain how the system will recognise an object. The type of sensor used. (LDR matrix, LIDAR, Web cam, etc). The resolution and matrix/ retina style. (Hexagonal, polar, etc)

And most importantly how the data stream will be stored/ searched and matched.

Edit: I'm not trying to shoot down your theory.  I presume you need help/ reassurance that you are on the correct path.  I can only give my opinions.

[keghn]

 High  BeautifulFlowers33.
 Yes it is a decision tree. But of What. And i will tell you and how do evolve it.
 Your bot pick a home base to start from. Next your bot picks a place to move to, and does it.
 Then you bot moves back or re tracks back in a round loop or a pull back.
 Next it picks a completely different place to move to and then move back to the same home base.
 Then the decision tree is used to assess on the two moves.
 like did one loop have more light for you solar panels? .............reward of energy gain.
 Was there a better place for a home base?..... reward of discover.
 Was the difference in the movements to consider them different?......  anti-reward of boring. Need strong and
greater evolution.
 Was the next move to radical and could not be compared to the other? ......  anti-reward of scared .


 You will need to encode the bots movement into memory of what it see and touches and what motors it used. From
its travels from home base to out in the wild and then back home base.
 If you can do this than this will be your first engrams path ways.

[LOCKSUIT]

(I'll reply to kehhn in 1 sec)

No no I love this disproving/etc. because I got this really down pat and in diagrams/etc. now. I forgot to add them, I can show you some stuff~



1. The eye saves a ongoing stream into the memory bar from left to right, then, continuously, pics or clips then go to match memory to pics/clips that 1) at least is similar and 2) picks the highest rank. The sense matching memory finds it by firing right to it on the first attempt by same spike strength pathways since our brain does not have a matching-up-to-matching-up system nor takes that long!



2. The story about the order. It must guess, then rank, so it uses this action only if it GOT the reward. This means:
Guess
Motors
Sensors
Triggers(rewards)
Memory

GM-STM
While the body&brain actually "looks" like STM-GM

The guess is actually fully guessed by the sensors sense being used, and the tweak of a action is done by it itself going through Guess. So if STMGM is done, it must through out the first sense gotten, so it saves the guesses&senses, guesses&senses, and so on. Also they get linked, even to next sets. And when matched-at they go up to reward triggers and back down to memory front and initiate.

When guessing it could do bad things like crush if was a huge car instead of a baby, you cannot shoot this at me like "when does it fold a towel", you're jumping the gun and throwing away how it really works while I do have how this all is done toooo. Seeing someone and knowing not to crush them is a action initiated/halted by neg-rank when see and it matched to a memory.



3. I'v written out a line of how it will be implemented onto hardware and is sequential or parrrel(all happening at the same time). Note that actions guessed are saved into that "mem-bar in the first box at bottom row" and then sent to motors, waiting for senses to be saved in the box above it and get linked, and all the actions&senses to come will be linked and therefore can be ranked even if sensed later after actions were done.



4. As for performing thousand actions for a task, first look at that a mouse learns a pull action to crawl/walk, then guesses once moves no more since stopping good rewards is a bad reward, then when it feels the same sense when its arm is back where started it does the pull action, and then the "good" ranked action to reverse. All 4 legs end up good. It ends up doing a few actions to places or things.

***As for us, in the advanced stage with linked visions to sounds, you get told grab sword which has linked action (it's now at memory bar front), then get told more actions!!! and all link in order after the first one at mem front!! That's your thinking! And the reason you faintly see it/hallucinate is because the sense matching memory matches not that much UNLIKE when a full sensed one coming from your eyes/hand which matches mem stronger! Otherwise without matching say nothing in mem, you'd see everything like a baby and nothing is understood or makes you do actions,



5. STICK FIGURES

When the video starts, they are generating actions, I know how c++ does this including the DICE BLOCK in the EV3 programming blocks software which is easy too.

The program takes the lil guys-----------and the one that got farrrr(trig reward like food)----------his tried actions------------are used into the neural network------------and tweaked a bit-----------then when a biggger reward like better food taste is gotten then this actions are ranked higher number and so when it sees it next time it matches mem to the strongest ranked sense&its-linked-actions as long as the sense is similar to the matching sense.

[LOCKSUIT]

(korrelan my reply to you is above, don't be scared, it's simple/short+fun[and needed for advancements])

Keghn, that kind of stuff is a add-on for the guessing and picking of actions, like always guess a new action/s~ I got how we return to places/etc. Adding built in ways or if roll over here and back and was it good isn't needed, and as for it "thinking" that I just said my AI can do that in the more advanced stage as mentioned above but not at first no.

[Korrelan]

I'm just concentrating on the eye for now. From what you have explained the same techniques are used to recognise objects as well as generate general intelligence.

The eye saves an ongoing stream into the memory bar from left to right, then, continuously, pics or clips then go to match memory to pics/clips that 1) at least is similar and 2) picks the highest rank.

Let’s say you have a seven pixel scan over a seven pixel wide frame.  This no were the resolution you would require to recognise an object but it brings us roughly to 7x7= 49 pixel array of sensors.  Again without taking into consideration pixel colour/ brightness that’s 49 x 48 x 47… x 3 x 2 possible images going into your memory list. Even limited to 10 frames a second and monochrome the storage requirements to save and rank each frame will quickly become un-workable. 

You don’t seem to understand the limitations of modern computers. We simply can’t search that amount of data in real time, that’s the point of AGI vision techniques and algorithms.

Just for kicks I wrote a quick simulation of your bot.  It only has 9 pixels of vision in the horizontal plain. You can see what the bot ‘sees’ across the bottom.  The size of each block depicts the distance away, like RADAR or SLAM and colour. It stores only unique images to memory.  The number in the bottom right is the number of unique 9 pixel wide image scans it takes in this short vid.  If the incoming image scan is already in memory it doesn’t save it.

In 47 seconds it collected 2000+ unique individual 9x1 pixel wide images.  That’s just the number of images, not the memory required to store them.



You are going to need an AGI algorithm/ technique/ system to ‘understand’ what the system is seeing.