computer eye cortex stuff with a corner tracker some half implementations

  • 31 Replies
  • 15823 Views
*

ranch vermin

  • Not much time left.
  • Terminator
  • *********
  • 947
  • Its nearly time!
(going for 3d now! :))

(latest viddie)
http://youtu.be/FIjqUevTjBo


It all started 1.6 years ago, when I wrote a grouping system, that shares shapes between whole entities->
pixel grouper->
http://www.youtube.com/watch?v=RtMou3RM01s#ws

and I progressed along and made another one.

grouper (like a restricted boltzmann machine) ->
http://www.youtube.com/watch?v=RgIkAj337dE#ws

then my best one of all time doing full colour and multiple levels grouping huge areas together->
http://www.youtube.com/watch?v=wIgfg1EFXLs#ws

The idea is, you share measurements between things in an object oriented hierarchy, with a single unified beginning (or end if you like) of an 'object',  a measurement could be pixel intesity, or adjacency information.   and everything shares with everything, these measurements,  the more measurements you share, the less cells (or 'classes' if you think about it object oriented) you need, and the smarter it is...  The real version does it off depth maps, and measures the viewspace distance between the pixels in the depth map.

These measurements are relative and invarient to spin, scaling and translation, and maybe even more when you go to 3d.    So you dont know exact positions of things, you just know distances between things... and youll come up with the class, and where it is as a VARIABLE.  not a solidly stored thing.

You spacially segment and track temporal conglomerations, using entropy, doing 2d is ALOT easier than 3d, but 3d is the best one,  you essentially are going to get video games out of ordinary video off youtube, for example.

Its animation is very similar to a markov chain chat bot,  so it doesnt make a hell lot of sense, but retains continuity.  you must pick only 1 symbol at a time,  because thats how chat bots work, and you want to make sure it works, so you be a little careful and only pick one action for a single class at a time.

You must fire through the same pipes for as many times as there is the same class in view,  that is the multiinstancability. (oo paradigm)

When i attempted to add the depth map, to record relative pixel distances, ill need one of these to track camera movement and produce maps best i can.
3d corner tracker->  http://www.youtube.com/watch?v=fnGs4hf3jO8#ws

And thats where im ended.   

Ive been programming since I was a kid, I learnt win32 when i was 19, made games ever since then but then  I started AI when i was 32,  and its been 2 years... and this is where im at.

I want to totally automate video game production, off ordinary video.  (my magnum opus.) XD but everyone should have their big important achievement,  cause if you dont think somethings important, then you disintrigrate and become sad.

WHAT I NEED->

* i need camera movement and depth map.

* the traits reconstruct the image, as well as classify it, they are topological distances. (2d or 3d its the same.)  so its kinda like a worldspace model.
  youve got a deapth map pixel truncation problem, its kinda antialiased how you make these traits.
  temporal traits will reconstruct movement, as well as classify movement.

* the topological information, will spread out the initial counts of things, you should be able to have overlapped classes, if it makes sense.

* sharing traits, will hit local optimas, so youll never get maximum compression, but the more you compress, the better off you are, intelligence&store saving, makes an inefficient thing a little more efficient.

* you can supply some segmentation help, from some instant algorythm, but entropy should do your conglomeration&separation temporally and
  spacially... although projected 3d visuals makes it more of a challenge for the algorythm... its a much tighter threshold.

« Last Edit: March 06, 2015, 03:28:50 am by ranch vermin »

*

Freddy

  • Administrator
  • **********************
  • Colossus
  • *
  • 6855
  • Mostly Harmless
Re: computer eye cortex stuff with a corner tracker some half implementations
« Reply #1 on: December 10, 2014, 02:39:37 pm »
This is complex stuff and above my head but interesting to read about...well the bits I understand.

Games made from video hey ? It's a new one on me, sounds cool.

*

ranch vermin

  • Not much time left.
  • Terminator
  • *********
  • 947
  • Its nearly time!
Re: computer eye cortex stuff with a corner tracker some half implementations
« Reply #2 on: December 10, 2014, 03:14:23 pm »
Thanks for the positivity.

If you consider a wiener filter as a form of memory system,  or the transition matrix of a markov chain, as a form of memory,  this just taking 1 dimension (one chat line, or one pixel), and the new way is making it dynamically dimensional.

*

ranch vermin

  • Not much time left.
  • Terminator
  • *********
  • 947
  • Its nearly time!
http://www.youtube.com/watch?v=XZbqs2NtT0Q#ws

Ive got my new system going, its a mininetwork running centre targetted on every pixel,   its not training yet,  but its counting the differences of whats there and outputting a permutation integer.    Only the first permutations are displayed, the rest is clamped 0, otherwise you cant tell its working from seeing it.

The interesting thing,  is the still parts of the screen, will report back the same permutation no matter where they are on the screen. (so you support scrolling straight away.)    In other words its 'translationally invariant."  Its like an emboss filter.

I was thinking this thing was going to be impossible until I came up with the systematic permutation idea (it doesnt train, it just puts the area of the screen through a process to label it),  then it flew,  but its not finished yet.

Then the idea goes, I then bind the frames together for the animating pixels, to track from moving piece to moving piece.
Im working on this now,  after half a year of theory!!! for the goal as it always was,  making games off video streams.

Theres no reason why this couldnt be used as a noun gatherer for a real ai, with a mechanical body.
« Last Edit: January 21, 2015, 04:45:04 pm by ranch vermin »

*

ranch vermin

  • Not much time left.
  • Terminator
  • *********
  • 947
  • Its nearly time!
Injected the screen palette into the surface ids

http://www.youtube.com/watch?v=Te4XiQCpvMc#ws

it hasnt got enough bits for the specificy it needs to tell surfaces apart,  so theres alot of colour sharing going on that shouldnt be there, it hasnt the capacity yet... i plan on improving the system to 30 bits,  it may make a huge difference from what it is now 24,  because 18 is way worse than 24, even tho its only 6 bits, and ive tested that.

Spin invarience is now implemented.  (all flip and mirror and 100 angle positions in 180 degrees.)   but it doesnt have the specific startoff it needs to start generalizing it logically.  (calling sameity.)

the difference between 32 bits, and 24 bits,   is 4096 8192x8192 textures for the id chart!!!   they are fully quantized, making one bit difference GINORMOUS!
« Last Edit: January 21, 2015, 08:12:08 pm by ranch vermin »

*

Ultron

  • Trusty Member
  • *******
  • Starship Trooper
  • *
  • 471
  • There are no strings on me.
I'm sorry but I didn't quite understand your goal. Are you attempting to create an algorithm to memorize patterns of shapes and later recognize various objects in 3D space ( from video ) ?
Software and Hardware developer, and everything in between.

*

ranch vermin

  • Not much time left.
  • Terminator
  • *********
  • 947
  • Its nearly time!
I'm sorry but I didn't quite understand your goal. Are you attempting to create an algorithm to memorize patterns of shapes and later recognize various objects in 3D space ( from video ) ?

Im going for a universal vision algorythm, its not like a little baby one that just fires a globe if something is in the view or not.
Just think of an amazing 2025 science fiction camera software application.

But its got a long way to go yet. :)

But... aiming for the clouds->

* perfectly segmented overlapped multiclassification.
* recovering a depth map.
* 3d optical flow.
* camera tracking (camera ego motion.)
* object oriented simulation.  (thats the fun bit,  it just models the animation of the objects it sees, and plays it back through correlation.)


It is programming itself off the sensor,  so it basicly works as a robots eye as well of course.   If you wanted to be poetic,  its feedback is its version of 'machine dreams'.


UPDATE:  got 28 bit id's to work, and it is better,  mortal kombat 2 comes out 70% perfect now. (much better than the vid i showed, which was ~23 bits.)

its running off a 16384x16384 STATE TEXTURE!!!  (and my video card has now started crashing its head off. XD)  so i think its at the point where i have to not fully store it, and do it the hard way, and just learn whats there.


I think im ringing my gtx980s neck, with this thing,  my computer might snuff out if I keep up the abuse without giving it long enough rests. :)

*

Ultron

  • Trusty Member
  • *******
  • Starship Trooper
  • *
  • 471
  • There are no strings on me.
Just avoid overclocking and your PC should be good :)


Anyway I hope you don't give up on this, your results seem impressive so far and eventually I'd love to help you test some done programs on actual robots :)
Keep up the good work!
Software and Hardware developer, and everything in between.

*

ranch vermin

  • Not much time left.
  • Terminator
  • *********
  • 947
  • Its nearly time!
Thanks Ultron, I guess it will be more impressive than just glitchy visual effects when ive got the segmenter + tracking running,  more than just machine reversal, (reconstitution of the input) But thats where all my first work was,  just making sure I could go into the system, and back out from the system.

EDIT:

Just added my next feature-  online web cam streaming.

https://www.youtube.com/watch?v=tXO5cl29DBE&feature=youtu.be

Its definitely more computervisiony doing that, means if i lag the system, it sees a sped up world, so it ruins his reflexes, and his entire speed judgement is off, unless you make it aware of the clock difference,  which definitely needs some thought.  Means you have to use T, when you compute things.

If you managed to get it fast enough, you could give him super high speed sampling, and he sees everything in slow motion. (good for a killer robot.)  But would take alot of processor power, for any intelligence at all.

what I learnt about this video decoding stuff

These days I recommend WMF (windows media foundation)  dont use direct show or avistream, they are both long gonners, and you can get mislead to learning an old library that doesn't work anymore, and download a heap of malware in the process, theres a bit of a security war with pirates and video streaming, and it makes things difficult,  its best to just stick with your own video,  decoding the formats you dont have is only to use something you didnt make.

It may be more boring just using your own media, but nothings stopping you from putting your movie director cap on, and learn a bit about special effects, lighting and scene composition, yourself.
« Last Edit: January 22, 2015, 08:12:51 pm by ranch vermin »

*

ranch vermin

  • Not much time left.
  • Terminator
  • *********
  • 947
  • Its nearly time!
The first step of my tracker is almost working.

http://www.youtube.com/watch?v=T4yHI5iQrc8#ws

improve:

http://www.youtube.com/watch?v=gSn6IIYCq2I#ws


now im going to track the vertices properly, and generate a true 3d lowpoly version of the camera video.
« Last Edit: January 26, 2015, 02:18:05 pm by ranch vermin »

*

Ultron

  • Trusty Member
  • *******
  • Starship Trooper
  • *
  • 471
  • There are no strings on me.
Re: computer eye cortex stuff with a corner tracker some half implementations
« Reply #10 on: January 26, 2015, 10:04:11 pm »
Your face reminds me of some 80's singer, possibly a rocker... Epic! xD

Back on topic - you are making progress very fast! I am still stuck with my project and back at learning stage. I would love to help you implement this technology one day, given my robotics career is successful x)
Software and Hardware developer, and everything in between.

*

Art

  • At the end of the game, the King and Pawn go into the same box.
  • Trusty Member
  • **********************
  • Colossus
  • *
  • 5865
Re: computer eye cortex stuff with a corner tracker some half implementations
« Reply #11 on: January 26, 2015, 10:44:33 pm »
Haha...lucky you didn't get slapped for that wrinkles remark! Gals don't like being reminded of those things or anything that makes them look less than a model!

Good going on your project. Vision is key to robots being able to explore, especially autonomously, provided the AI is usefully employed! Otherwise, it's a pile of metal, gears, plastic, electronics and wiring.
In the world of AI, it's the thought that counts!

*

Ultron

  • Trusty Member
  • *******
  • Starship Trooper
  • *
  • 471
  • There are no strings on me.
Re: computer eye cortex stuff with a corner tracker some half implementations
« Reply #12 on: January 26, 2015, 11:02:39 pm »
Eye! The software is where the true ingenuity comes in! Hardware is generally the same with all robots today - motors, hydraulics, computer, Ultrasound sensors. Hardware advances at a slower pace, and one man alone cannot make a large impact in that field (unless of course he is the reincarnation of Tesla). Software on the other hand, advances at a much faster pace and it's the way for the big minds of our time to show off their skills.


But today every field has advanced so much that one man can achieve nothing on his own. I also believe that one day a group of fellas from this forum will join on a collaborative effort, join their projects and make the world drool.
Software and Hardware developer, and everything in between.

*

ranch vermin

  • Not much time left.
  • Terminator
  • *********
  • 947
  • Its nearly time!
Re: computer eye cortex stuff with a corner tracker some half implementations
« Reply #13 on: January 27, 2015, 01:39:31 am »
Thanks guys!    Im really glad this one came through.    On a collaborative effort, that could be fun. :)

Its a pixel parallel filter in this sandwich->

a) corner filter
b) dilation filter
(then repeat - this swaps the dilations onto the triangle centres)
c) corner filter 2
d) dilation filter 2
e) point in triangle filter

Ive seen a few demos here and there on the internet which have already done the things im planning on doing next,   Amusingly I think everything I need to make this a.i. of mine has already been developed if I scrounge around the internet hard enough and meld it all together.


http://www.youtube.com/watch?v=Z6HFRN9Q6wk#ws



I just had another think  and I think full 3d is possible from monocular video (including vintage 1800's footage)->

The real deal->


CORNER DETECTION (these are the base tokens.)
STEP 1. corner detector basis.
STEP 2. make an ordinary corner to corner matcher.
STEP 3. add the machine learning to the corners. (must discuss tracker problems here.)

TESSELATION MEMORY
STEP 1. raster the known lines on the screen.
STEP 2. modify the tesselation filter to not go over lines already made.
STEP 3. keep adding to your collected lines.

THEN THE DISCONTINUITY CORNER PROBLEM->
* a line was made at a discontinuity.
* a shape is introduced to make a discontinuity, on a non discontinuity.

2D TO 3D DISTANCE
* you have a million 2d samples, and you need a 3d distance.  say you know the projection matrix.
* the secret is adjacency,  then yes theres errors on softbodies but on a whole it should work, because its relative edge.


SPACIAL&TEMPORAL PREDICTOR
STEP 1. scoped corner predictor.
STEP 2. think of how to do animation.



« Last Edit: January 28, 2015, 03:30:43 am by ranch vermin »

*

ranch vermin

  • Not much time left.
  • Terminator
  • *********
  • 947
  • Its nearly time!
Re: computer eye cortex stuff with a corner tracker some half implementations
« Reply #14 on: January 30, 2015, 11:25:04 am »
After that post of gibberish I just implemented the beginnings of the cool bit (actual vertex tracking)->

http://www.youtube.com/watch?v=klDGQ9VWrTE#ws

But its only early days yet -  If you want to see what its like when its closer to finished look up Zdenek Predator tracker computer vision,  the google tech talk is where he shows the 3d potential.    And where all the ideas for this came from.

 


OpenAI Speech-to-Speech Reasoning Demo
by ivan.moony (AI News )
Today at 01:31:53 pm
Say good-bye to GPUs...
by MikeB (AI News )
March 23, 2024, 09:23:52 am
Google Bard report
by ivan.moony (AI News )
February 14, 2024, 04:42:23 pm
Elon Musk's xAI Grok Chatbot
by MikeB (AI News )
December 11, 2023, 06:26:33 am
Nvidia Hype
by 8pla.net (AI News )
December 06, 2023, 10:04:52 pm
How will the OpenAI CEO being Fired affect ChatGPT?
by 8pla.net (AI News )
December 06, 2023, 09:54:25 pm
Independent AI sovereignties
by WriterOfMinds (AI News )
November 08, 2023, 04:51:21 am
LLaMA2 Meta's chatbot released
by 8pla.net (AI News )
October 18, 2023, 11:41:21 pm

Users Online

318 Guests, 0 Users

Most Online Today: 346. Most Online Ever: 2369 (November 21, 2020, 04:08:13 pm)

Articles