[LOCKSUIT]

I would like to know if anyone here knows the generic explanation of how those quad/spider robots learn their crawl?

I do get, that, at an instant moment, all motors receive randomly generated actions, then it gets a high accelometer signal, then rewards the actions, and does them on recognition.

But what I'm asking is the sequence of action sets. It's not just 1 set of motor actions. First 1 leg moves, then its body, then its other, etc.

So, am I right to say that each set of motor actions is either: rewarded, or rewarded by leading up to a rewarding?

That would explain how that AI went around and kicked the soccer ball into goal net.



I have another question. Are these algorithms generating actions, then doing the actions, then tweaking the generators's parameters? Or are these algorithms generating actions, then doing the actions plus saving them, then tweaking the actions when pass the generators? The later allows the generators to not output only 1 type of actions.

[LOCKSUIT]

I think I know how....

............they are massively cheating by using the reward "how far it gets in ex. 5 seconds".

That would (bad though) skip saving actions, rewarding ones by if lead up to rewards, doing each set of the crawl on cue, etc

But the best "play" being tweaked doesn't explain why they always re-use their own-unique 4 legged-move crawl 1 2 3 4 1 2 3 4...dynamic cut-off?

[Korrelan]

If you’re referring to the blender animation of the four legged creatures learning to crawl then yes… they are probably using the greatest distance travelled in the allotted time as the ‘reward’.  Though ‘reward’ is probably an inaccurate description; it’s based on a genetic algorithm and is a variation on the old method of ‘motor babbling’.

The basic idea is that each bot has an initial random motor/ servo plan that moves its legs in a set fashion/ sequence. 

The servo plans/ maps of the two bots (or single) that travel the furthest are combined and several slight random generations are produced based on the winners.  This means that all the new bots have the best bits of the bots that travelled the furthest plus a random factor that ‘could’ make them more efficient. Over thousands of generations this technique can produce efficient locomotion; though not usually as you would imagine lol.

Nature/ evolution has had millions of years to refine the gaits of animals and loads of other factors besides the distance travelled has influenced their efficient movements.

 

[LOCKSUIT]

So to sum up what you said, if we imagine only 1 crawler for simplification, it receives a generated/tweaked set of motor actions, reward, repeat?

But wait, they use all 4 legs at the SAME time, and the crawl is a repeated sequence, ex. 4 legs do this, then 4 legs do this, then 4 legs do this, REPEAT.........How?

If that's the reward they use, then they are capturing plays of generated/tweaked sets, but cutting off the play so the sequence is long enough.

Also is the algorithm using a spatial map to see which areas are blossoming?

[Korrelan]

As usual there are many methods/ techniques to this kind of system.

It receives a generated/tweaked set of motor actions, reward, repeat?

It looks to me that this is a genetic algorithm based approach… and there is NO REWARD lol.  Unless you class not being deleted as a reward… the traits of the winner(s) are cloned and randomly slightly modified.  Walking efficiently is an emergent property from the evolution of the genetic algorithm.

But wait, they use all 4 legs at the SAME time

A motor map can move as many servos as required.  There are loads of different formats and servos can be linked/ timed either independently or as a complete schema.  A motor map usually comprises of the motor number, duration of action, etc. that would play in a loop. Some systems have a sensory map as well and complex interplay between the various maps.

Again… THERE IS NO REWARD… required in this type of simulation. lol

 

[LOCKSUIT]

To know which to delete requires a reward. You can't get crawl actions to move it without reward.

Building houses. Eating. Mating. Picking back. Going after football games. Fine dinning.

[keghn]

First Steps of Machine Life.

Step one. Find a pattern loop and then hit it with a output.

 First pattern loop is when you bot first turn on and looks at a wall and nothing is moving. The position of legs, arms and
wheels are not moving. so they are the same position every second. So the values of all pixels are the same every
second along with values of the wheel position and GPS. Fist pattern loop found. That is not hard.

 Step two every pattern loop gets its own a burn out register.

 When a pattern is repeated too many time this register gets incremented. Too high of a value is seen as pain.

 Output is a pulse as like hitting a piano key. One pulse can be sent to the wheel motor or to the arm motor.


 Step three get back to the starting of the loop. Or "getting back to home base".

 When you pulse the arm motor it moves and then comes back to where it started. no big deal. but it get a reward for finding
a new repeat loop. But it can only do it so many time before burn out register accumulates a high value.
 When the wheel motor is pulsed the GPS position changes and wheel position changes, also. The bot gets a minor
reward for finding new move, Called the launch. But until it find it way back to its Starting GPS position the
loop is not complete. It can only be completed by trial and error of by moving back and forth, Or rolling forward in
complete circle. This can take a long time at first. Take two year for a human baby to walk.

Burn out register are down counted when its pattern loops is not used.

A continuous pulse to the motors is a bunch of pulse in a row.

A strong pulse is 10 pulse sent at the same time.

 For a completely free will bot it has to find out what computer instruction pulse what motor by trial and error. It does not
know. The bot remembers everything, records everything in a video stream including wheel positions, arm positions, audio,
and output pulses.

[Korrelan]

http://rednuht.org/genetic_walkers/

http://matthias-rolf.blogspot.co.uk/p/goal-babbling.html

 

[keghn]

 Eye can be lightly spring loaded so it comes back to the same position after a output pulse.

[keghn]

Phototaxis demonstration: 




Josh Bongard:

https://www.youtube.com/channel/UCKCmKOwVkoAh2NDp0T-Ovng/videos

[keghn]

Unit 2, Topic 30, State Space Diagram and More Complexity: 

[LOCKSUIT]

Yes large state spaces/dimensions.

Think about these 2 things:

1) All possibilities/states in a finite Earth sized sphere.
2) All possible ways to search;

Ex. linear, exponential, random, tweak (ex. 22% random), specific desired sequence of actions / technology you want, probabilitic by successes & failures and success & failure rates.

You may think it would forever, but there are genius ways to search better to find ANYTHING you want fast.

[keghn]

SPYNDRA ROBOT: 
http://www.creativemachineslab.com/spyndra.html