[Ultron]

Did you mean this?




You are right the potential is truly amazing, and it takes on 3D pretty well. This could have countless applications in robotics and AI - and it is all demonstrated through a webcam!
Also, I see they have improved their program since, can't wait to see a new demo!


http://tldvision.com/


Although I could not get my hands on a demo of TLD 2.0, I found that they have released the source for TLD 1.0 (that should be fun enough).
https://github.com/zk00006/OpenTLD


Regarding the concept, I quote:


"Tracking estimates the object location as long as the object is visible. During tracking all observed patterns of the object are used to learn an object detector."

Estimating the objects location is definitely a good idea towards assisting object tracking. And the second sentence shows off a truly amazing idea. That may actually be how we learn to better recognize a face - we know where the face is, and we just observe it while we can see it. In the meantime it moves and we get to see it from different sides, eventually even forming a 3D representation of it and we would thus later be able to recognize it more easily.


How long do you think before you can achieve something like this, ranch?

[ranch vermin]

Hey thanks for the post and link, so I dont have to do it.     One thing id like to add,  is you can accidentally come up with this, if you try to make an insect compound eye - as in each segment responds to a pattern and indicates an id/token.   then repeat the network on every ommatidium. (making it so they learn each others patterns)  and you gain translational invarience,  which is what predator, and all the rest of the best rgb web cam eyes use.  (say if you dont want to use a kinect, you dont have to.)

And your question -> This way is superior,  I believe it will go all the way to full 3d reconstruction.

Its hardly running, (ive only just got this working, it was a good feeling getting it running.)  but every now and again, itll latch onto the video.

[Ultron]

Judge Dredd! Nice haha... Well now I atleast know exactly what you are doing... Hopefully I'l come up with some ideas soon so I can help out a bit.

As for reconstructing 3D objects... That takes input from two cameras (OK easy) but also takes into account shadows (professional filmography here). But you are trying to do this with 1 camera, so I assume you take into account shadows and object movement (if it is getting smaller it is getting further away) ?

I am not sure as to how humans triangulate distance and imagine 3D space naturally, I would say we compare the average size of an item with what we see but that doesn't work..

[ranch vermin]

Its not done with triangulation exactly.

Im going to be finding relative distances between springs, not exact positions,  so that way I can discover moving objects as well as still. (The basic monocular problem, cant find anything moving.)


this is just before I go and learn the mesh off the tesselations,  then youll really get the 'virtualized' effect.





EDIT->

Ive fixed up a system a damn lot,  now its 256 bit descriptors and its running quite well,  although its still a bit slidey on tricky flat colour spots.
The face pretty much comes out tho,  as I would expect, because faces triangulate depth positions stereo fairly well,  and the same goes for optical flow, they both use rgb brightness as an input.

[ranch vermin]

improved the code.



look in description for algorythm
https://www.youtube.com/watch?v=6pTf_lYeoqI&feature=youtu.be

[ranch vermin]

Been a while, but now i've now got a theory down to implementation for 3d off an ordinary web cam, that should destroy anyone to nows attempt at camera tracking, and compositing and everything.    sound arrogant?   Possibly I was arrogant for even attempting it, but I was mistaken in how important it would make me to others,  I was silly for even wanting to be important.  I should be embarressed.

1. calibrate your cameras field of vision, to an exact ray step per pixel.

2. I have a formula,  where I can brute force the 3d distance of an edge looking at so many samples of it over time. (im guessing about 5 frames, to uneven the scores.)

3. I use the same formula, to then take this 3d distance and brute force real viewspace coordinates.

4. the sneaky trick, is I then reduct these 3d positions using the original projection   So I just keep the 3d position closest to the 2d position.

5. then I can reproject any orientation, and then its all possible after that.   


Ill be back in a couple of days TOPS!   then itll be fully 3d,  and you cant guess what I can do with this thing. 


Just think  -  first there was digital zoom,  now theres complete digital camera repositioning!! XD

[Ultron]

Can't wait to (hopefully) get my hands on this... I can easily incorporate it into my thrifty AGI robot concept / design. Currently looking to buy cheap camera modules from laptops which can be easily hacked and cheap.

[ranch vermin]

Indeed its just off raw colours.
Sure you can have it ultron cause weve been talking,  but this algorythm is copyright,  except in certain circumstances.  Means if I give it to you, your not allowed to give it to anyone, or I sue.   No shit.

Took a long yonder in my head, and got the spirit back to get back typing, realizing the tracker is horrible and must be improved before any 3d calculation takes place.
its a big truck load into it, the 1kilobit name per pixel, I couldnt get it any faster than this, but the mesh is really solid, but it does still stuff up. I do have the 3d brute force type algorythm organized, but it may be a case of not going very fun speed, so far.

Once 3d is done, I then predict the back side of things, and do the proper segmentation, required to wrap around objects.  With a bit of luck, itll be a pretty zany application.

[Ultron]

Fear not as I doubt I would ever understand your algorithm

The only thing I am interested in is combining some cool concepts and technologies such as your own to create a real working artificial mind - something you do not need billions of dollars and a huge computer cluster for, only a bit of brains (punny, isn't it?).

Anyway, would you mind discussing your idea on how your program would predict the 'invisible' side of an entity? I have had several hypothesis about how humans do this.

[Art]

It didn't appear as if this 1024 one was as defined as your earlier 256 one. Then again, maybe it's me.

The music is um...interesting.

[Freddy]

Impressive, I could recognise your face and hands quite well in that one.

[ranch vermin]

I had a look at the code again,   and I got it running faster.   It always gets better the more I sit and tinker with something.

Ultron,  I didnt believe I could get one of these video nets running on one gpu until very recently.  The finished one will work on 4 or so at once, in sli.

Art,  that music is my favorite for doing this mechanization stuff with.  Futuristic drum and bass really puts me in the mood for making a robot. hehe.  And trust me its definitely better with 1024 bit descriptor,  I can vouch for that now I looked at it for ages,  the surface positions arent stable without it.

How to see the backsides of things?   Very simple,  Just as a markov chain word predictor will predict words,  you just have to think in 3d space instead, and wrap around the other side as if it was a sentence of words.   Just saying generally.   

So if you see a foot,  you have a person.   

If you see a basketball, maybe theres a kid.

If its done statistically and stochastically it may be quite amusing watching it come up with random scenes.


Bringing back animation, is as if Ive put all the capture into lots of backpropagator's, doing everything at once, and  I bring it back like the animation in GTA V.  But all done in a "greedy fell swooping fashion"...  which is what my middle name phrase is,  definitely not originality.
Which interestingly I dont think there is much originality required in the ai field, (especially thinking about computer vision.) its all a whole load of already done and thought about stuff that needs to be put together and implemented.     Its just something you can do for yourself, so you dont have to pay anyone.
But not saying there isnt more to theorize, especially for less artificial vision,  which this is very artificial - and has trouble with translucency.

I wonder how good some of the hollywood visual effects mocap programmers get their algorythms!   I think it must be better than mine, some of the movies are pretty schmik.  This 'universal capture' stuff has been out since the matrix movie with Keanu,  but they did it in a camera room based system,  and I guess these days its all about getting it work out of just one ordinary camera.

[Ultron]

Seriously - a 4 nVidia SLI Graphic Card system? I thought making a custom machine with 2 cross-fire cards was expensive enough, but this?

[ranch vermin]

Art-  Just correcting myself,  256 actually is when it starts to get good, actually going past that only improves a little, but its worth it.

Just got to wait for the 980 to cost $400  then ill upgrade 2 more of them, and have 3 -  in about 3 or so years.
Yeh I know im an nvidia idiot,  remember back when intel cpus were so cool but so expensive.  Im just an idiot for buying it,  but they do go kinda smooth, but I know I wish I grew a brain and forgot about nvidia,  I HATE CUDA,  I only like direct compute anyway.  CUDA takes directx shader code and makes it run ~8 times slower,  I cant be bothered learning it, Its more for experts.

The track is nearly ready for me to go try out my 3d algorythm, but not quite yet.

One could think its specular which is ruining your track, but One could think its actually when two bodies are pulling apart. (which even happens on the nose, as it points out and rotates im thinking, also the side of the head, also the lips when they open.)  The tracker doesnt know what of the bodies it belongs to.   The same thing happens when one track's a 2d platformer,  near the tile ground, it doesnt know to follow the tile or follow the character.  And I think that is the main issue,  not translucency/reflection.

https://www.youtube.com/watch?v=XCk0e-YvXXs&feature=youtu.be

The obvious solution is I need a segmenter which makes sure it doesnt read the descriptor past the segment its centred apon.  So thats what i'm doing next.

[Ultron]

Hey ranch, let me know when you start running the final tests. I'd like to suggest a few algorithms or rather, concepts regarding artificial visions... I'l just let you know when they are mature enough.