Ai Dreams Forum
Member's Experiments & Projects => AI Programming => Topic started by: ranch vermin on December 09, 2014, 04:41:39 pm
-
(going for 3d now! :))
(latest viddie)
http://youtu.be/FIjqUevTjBo (http://youtu.be/FIjqUevTjBo)
It all started 1.6 years ago, when I wrote a grouping system, that shares shapes between whole entities->
pixel grouper->
http://www.youtube.com/watch?v=RtMou3RM01s#ws (http://www.youtube.com/watch?v=RtMou3RM01s#ws)
and I progressed along and made another one.
grouper (like a restricted boltzmann machine) ->
http://www.youtube.com/watch?v=RgIkAj337dE#ws (http://www.youtube.com/watch?v=RgIkAj337dE#ws)
then my best one of all time doing full colour and multiple levels grouping huge areas together->
http://www.youtube.com/watch?v=wIgfg1EFXLs#ws (http://www.youtube.com/watch?v=wIgfg1EFXLs#ws)
The idea is, you share measurements between things in an object oriented hierarchy, with a single unified beginning (or end if you like) of an 'object', a measurement could be pixel intesity, or adjacency information. and everything shares with everything, these measurements, the more measurements you share, the less cells (or 'classes' if you think about it object oriented) you need, and the smarter it is... The real version does it off depth maps, and measures the viewspace distance between the pixels in the depth map.
These measurements are relative and invarient to spin, scaling and translation, and maybe even more when you go to 3d. So you dont know exact positions of things, you just know distances between things... and youll come up with the class, and where it is as a VARIABLE. not a solidly stored thing.
You spacially segment and track temporal conglomerations, using entropy, doing 2d is ALOT easier than 3d, but 3d is the best one, you essentially are going to get video games out of ordinary video off youtube, for example.
Its animation is very similar to a markov chain chat bot, so it doesnt make a hell lot of sense, but retains continuity. you must pick only 1 symbol at a time, because thats how chat bots work, and you want to make sure it works, so you be a little careful and only pick one action for a single class at a time.
You must fire through the same pipes for as many times as there is the same class in view, that is the multiinstancability. (oo paradigm)
When i attempted to add the depth map, to record relative pixel distances, ill need one of these to track camera movement and produce maps best i can.
3d corner tracker-> http://www.youtube.com/watch?v=fnGs4hf3jO8#ws (http://www.youtube.com/watch?v=fnGs4hf3jO8#ws)
And thats where im ended.
Ive been programming since I was a kid, I learnt win32 when i was 19, made games ever since then but then I started AI when i was 32, and its been 2 years... and this is where im at.
I want to totally automate video game production, off ordinary video. (my magnum opus.) XD but everyone should have their big important achievement, cause if you dont think somethings important, then you disintrigrate and become sad.
WHAT I NEED->
* i need camera movement and depth map.
* the traits reconstruct the image, as well as classify it, they are topological distances. (2d or 3d its the same.) so its kinda like a worldspace model.
youve got a deapth map pixel truncation problem, its kinda antialiased how you make these traits.
temporal traits will reconstruct movement, as well as classify movement.
* the topological information, will spread out the initial counts of things, you should be able to have overlapped classes, if it makes sense.
* sharing traits, will hit local optimas, so youll never get maximum compression, but the more you compress, the better off you are, intelligence&store saving, makes an inefficient thing a little more efficient.
* you can supply some segmentation help, from some instant algorythm, but entropy should do your conglomeration&separation temporally and
spacially... although projected 3d visuals makes it more of a challenge for the algorythm... its a much tighter threshold.
-
This is complex stuff and above my head but interesting to read about...well the bits I understand.
Games made from video hey ? It's a new one on me, sounds cool.
-
Thanks for the positivity.
If you consider a wiener filter as a form of memory system, or the transition matrix of a markov chain, as a form of memory, this just taking 1 dimension (one chat line, or one pixel), and the new way is making it dynamically dimensional.
-
http://www.youtube.com/watch?v=XZbqs2NtT0Q#ws (http://www.youtube.com/watch?v=XZbqs2NtT0Q#ws)
Ive got my new system going, its a mininetwork running centre targetted on every pixel, its not training yet, but its counting the differences of whats there and outputting a permutation integer. Only the first permutations are displayed, the rest is clamped 0, otherwise you cant tell its working from seeing it.
The interesting thing, is the still parts of the screen, will report back the same permutation no matter where they are on the screen. (so you support scrolling straight away.) In other words its 'translationally invariant." Its like an emboss filter.
I was thinking this thing was going to be impossible until I came up with the systematic permutation idea (it doesnt train, it just puts the area of the screen through a process to label it), then it flew, but its not finished yet.
Then the idea goes, I then bind the frames together for the animating pixels, to track from moving piece to moving piece.
Im working on this now, after half a year of theory!!! for the goal as it always was, making games off video streams.
Theres no reason why this couldnt be used as a noun gatherer for a real ai, with a mechanical body.
-
Injected the screen palette into the surface ids
http://www.youtube.com/watch?v=Te4XiQCpvMc#ws (http://www.youtube.com/watch?v=Te4XiQCpvMc#ws)
it hasnt got enough bits for the specificy it needs to tell surfaces apart, so theres alot of colour sharing going on that shouldnt be there, it hasnt the capacity yet... i plan on improving the system to 30 bits, it may make a huge difference from what it is now 24, because 18 is way worse than 24, even tho its only 6 bits, and ive tested that.
Spin invarience is now implemented. (all flip and mirror and 100 angle positions in 180 degrees.) but it doesnt have the specific startoff it needs to start generalizing it logically. (calling sameity.)
the difference between 32 bits, and 24 bits, is 4096 8192x8192 textures for the id chart!!! they are fully quantized, making one bit difference GINORMOUS!
-
I'm sorry but I didn't quite understand your goal. Are you attempting to create an algorithm to memorize patterns of shapes and later recognize various objects in 3D space ( from video ) ?
-
I'm sorry but I didn't quite understand your goal. Are you attempting to create an algorithm to memorize patterns of shapes and later recognize various objects in 3D space ( from video ) ?
Im going for a universal vision algorythm, its not like a little baby one that just fires a globe if something is in the view or not.
Just think of an amazing 2025 science fiction camera software application.
But its got a long way to go yet. :)
But... aiming for the clouds->
* perfectly segmented overlapped multiclassification.
* recovering a depth map.
* 3d optical flow.
* camera tracking (camera ego motion.)
* object oriented simulation. (thats the fun bit, it just models the animation of the objects it sees, and plays it back through correlation.)
It is programming itself off the sensor, so it basicly works as a robots eye as well of course. If you wanted to be poetic, its feedback is its version of 'machine dreams'.
UPDATE: got 28 bit id's to work, and it is better, mortal kombat 2 comes out 70% perfect now. (much better than the vid i showed, which was ~23 bits.)
its running off a 16384x16384 STATE TEXTURE!!! (and my video card has now started crashing its head off. XD) so i think its at the point where i have to not fully store it, and do it the hard way, and just learn whats there.
I think im ringing my gtx980s neck, with this thing, my computer might snuff out if I keep up the abuse without giving it long enough rests. :)
-
Just avoid overclocking and your PC should be good :)
Anyway I hope you don't give up on this, your results seem impressive so far and eventually I'd love to help you test some done programs on actual robots :)
Keep up the good work!
-
Thanks Ultron, I guess it will be more impressive than just glitchy visual effects when ive got the segmenter + tracking running, more than just machine reversal, (reconstitution of the input) But thats where all my first work was, just making sure I could go into the system, and back out from the system.
EDIT:
Just added my next feature- online web cam streaming.
https://www.youtube.com/watch?v=tXO5cl29DBE&feature=youtu.be (https://www.youtube.com/watch?v=tXO5cl29DBE&feature=youtu.be)
Its definitely more computervisiony doing that, means if i lag the system, it sees a sped up world, so it ruins his reflexes, and his entire speed judgement is off, unless you make it aware of the clock difference, which definitely needs some thought. Means you have to use T, when you compute things.
If you managed to get it fast enough, you could give him super high speed sampling, and he sees everything in slow motion. (good for a killer robot.) But would take alot of processor power, for any intelligence at all.
what I learnt about this video decoding stuff
These days I recommend WMF (windows media foundation) dont use direct show or avistream, they are both long gonners, and you can get mislead to learning an old library that doesn't work anymore, and download a heap of malware in the process, theres a bit of a security war with pirates and video streaming, and it makes things difficult, its best to just stick with your own video, decoding the formats you dont have is only to use something you didnt make.
It may be more boring just using your own media, but nothings stopping you from putting your movie director cap on, and learn a bit about special effects, lighting and scene composition, yourself.
-
The first step of my tracker is almost working.
http://www.youtube.com/watch?v=T4yHI5iQrc8#ws (http://www.youtube.com/watch?v=T4yHI5iQrc8#ws)
improve:
http://www.youtube.com/watch?v=gSn6IIYCq2I#ws (http://www.youtube.com/watch?v=gSn6IIYCq2I#ws)
now im going to track the vertices properly, and generate a true 3d lowpoly version of the camera video.
-
Your face reminds me of some 80's singer, possibly a rocker... Epic! xD
Back on topic - you are making progress very fast! I am still stuck with my project and back at learning stage. I would love to help you implement this technology one day, given my robotics career is successful x)
-
Haha...lucky you didn't get slapped for that wrinkles remark! Gals don't like being reminded of those things or anything that makes them look less than a model!
Good going on your project. Vision is key to robots being able to explore, especially autonomously, provided the AI is usefully employed! Otherwise, it's a pile of metal, gears, plastic, electronics and wiring.
-
Eye! The software is where the true ingenuity comes in! Hardware is generally the same with all robots today - motors, hydraulics, computer, Ultrasound sensors. Hardware advances at a slower pace, and one man alone cannot make a large impact in that field (unless of course he is the reincarnation of Tesla). Software on the other hand, advances at a much faster pace and it's the way for the big minds of our time to show off their skills.
But today every field has advanced so much that one man can achieve nothing on his own. I also believe that one day a group of fellas from this forum will join on a collaborative effort, join their projects and make the world drool.
-
Thanks guys! Im really glad this one came through. On a collaborative effort, that could be fun. :)
Its a pixel parallel filter in this sandwich->
a) corner filter
b) dilation filter
(then repeat - this swaps the dilations onto the triangle centres)
c) corner filter 2
d) dilation filter 2
e) point in triangle filter
Ive seen a few demos here and there on the internet which have already done the things im planning on doing next, Amusingly I think everything I need to make this a.i. of mine has already been developed if I scrounge around the internet hard enough and meld it all together.
http://www.youtube.com/watch?v=Z6HFRN9Q6wk#ws (http://www.youtube.com/watch?v=Z6HFRN9Q6wk#ws)
I just had another think and I think full 3d is possible from monocular video (including vintage 1800's footage)->
The real deal->
CORNER DETECTION (these are the base tokens.)
STEP 1. corner detector basis.
STEP 2. make an ordinary corner to corner matcher.
STEP 3. add the machine learning to the corners. (must discuss tracker problems here.)
TESSELATION MEMORY
STEP 1. raster the known lines on the screen.
STEP 2. modify the tesselation filter to not go over lines already made.
STEP 3. keep adding to your collected lines.
THEN THE DISCONTINUITY CORNER PROBLEM->
* a line was made at a discontinuity.
* a shape is introduced to make a discontinuity, on a non discontinuity.
2D TO 3D DISTANCE
* you have a million 2d samples, and you need a 3d distance. say you know the projection matrix.
* the secret is adjacency, then yes theres errors on softbodies but on a whole it should work, because its relative edge.
SPACIAL&TEMPORAL PREDICTOR
STEP 1. scoped corner predictor.
STEP 2. think of how to do animation.
-
After that post of gibberish I just implemented the beginnings of the cool bit (actual vertex tracking)->
http://www.youtube.com/watch?v=klDGQ9VWrTE#ws (http://www.youtube.com/watch?v=klDGQ9VWrTE#ws)
But its only early days yet - If you want to see what its like when its closer to finished look up Zdenek Predator tracker computer vision, the google tech talk is where he shows the 3d potential. And where all the ideas for this came from.
-
Did you mean this?
Predator: Camera That Learns (http://www.youtube.com/watch?v=1GhNXHCQGsM#)
You are right the potential is truly amazing, and it takes on 3D pretty well. This could have countless applications in robotics and AI - and it is all demonstrated through a webcam!
Also, I see they have improved their program since, can't wait to see a new demo!
http://tldvision.com/ (http://tldvision.com/)
Although I could not get my hands on a demo of TLD 2.0, I found that they have released the source for TLD 1.0 (that should be fun enough).
https://github.com/zk00006/OpenTLD (https://github.com/zk00006/OpenTLD)
Regarding the concept, I quote:
"Tracking estimates the object location as long as the object is visible. During tracking all observed patterns of the object are used to learn an object detector."
Estimating the objects location is definitely a good idea towards assisting object tracking. And the second sentence shows off a truly amazing idea. That may actually be how we learn to better recognize a face - we know where the face is, and we just observe it while we can see it. In the meantime it moves and we get to see it from different sides, eventually even forming a 3D representation of it and we would thus later be able to recognize it more easily.
How long do you think before you can achieve something like this, ranch?
-
Hey thanks for the post and link, so I dont have to do it. One thing id like to add, is you can accidentally come up with this, if you try to make an insect compound eye - as in each segment responds to a pattern and indicates an id/token. then repeat the network on every ommatidium. (making it so they learn each others patterns) and you gain translational invarience, which is what predator, and all the rest of the best rgb web cam eyes use. (say if you dont want to use a kinect, you dont have to.)
And your question -> This way is superior, I believe it will go all the way to full 3d reconstruction.
Its hardly running, (ive only just got this working, it was a good feeling getting it running.) but every now and again, itll latch onto the video.
http://www.youtube.com/watch?v=xs1Iy6TkPkU#ws (http://www.youtube.com/watch?v=xs1Iy6TkPkU#ws)
-
Judge Dredd! Nice haha... Well now I atleast know exactly what you are doing... Hopefully I'l come up with some ideas soon so I can help out a bit.
As for reconstructing 3D objects... That takes input from two cameras (OK easy) but also takes into account shadows (professional filmography here). But you are trying to do this with 1 camera, so I assume you take into account shadows and object movement (if it is getting smaller it is getting further away) ?
I am not sure as to how humans triangulate distance and imagine 3D space naturally, I would say we compare the average size of an item with what we see but that doesn't work..
-
Its not done with triangulation exactly. ;)
Im going to be finding relative distances between springs, not exact positions, so that way I can discover moving objects as well as still. (The basic monocular problem, cant find anything moving.)
this is just before I go and learn the mesh off the tesselations, then youll really get the 'virtualized' effect. :)
http://www.youtube.com/watch?v=lLT-6qsUiQI#ws (http://www.youtube.com/watch?v=lLT-6qsUiQI#ws)
EDIT->
Ive fixed up a system a damn lot, now its 256 bit descriptors and its running quite well, although its still a bit slidey on tricky flat colour spots.
The face pretty much comes out tho, as I would expect, because faces triangulate depth positions stereo fairly well, and the same goes for optical flow, they both use rgb brightness as an input.
https://www.youtube.com/watch?v=q_6ytN1ybmI (https://www.youtube.com/watch?v=q_6ytN1ybmI)
https://www.youtube.com/watch?v=zlITUCnxwWA (https://www.youtube.com/watch?v=zlITUCnxwWA)
-
improved the code.
http://www.youtube.com/watch?v=BTiL9QKv7fA#ws (http://www.youtube.com/watch?v=BTiL9QKv7fA#ws)
look in description for algorythm
https://www.youtube.com/watch?v=6pTf_lYeoqI&feature=youtu.be (https://www.youtube.com/watch?v=6pTf_lYeoqI&feature=youtu.be)
-
Been a while, but now i've now got a theory down to implementation for 3d off an ordinary web cam, that should destroy anyone to nows attempt at camera tracking, and compositing and everything. :) sound arrogant? Possibly I was arrogant for even attempting it, but I was mistaken in how important it would make me to others, I was silly for even wanting to be important. I should be embarressed.
1. calibrate your cameras field of vision, to an exact ray step per pixel.
2. I have a formula, where I can brute force the 3d distance of an edge looking at so many samples of it over time. (im guessing about 5 frames, to uneven the scores.)
3. I use the same formula, to then take this 3d distance and brute force real viewspace coordinates.
4. the sneaky trick, is I then reduct these 3d positions using the original projection :) So I just keep the 3d position closest to the 2d position.
5. then I can reproject any orientation, and then its all possible after that.
Ill be back in a couple of days TOPS! then itll be fully 3d, and you cant guess what I can do with this thing. :)
Just think - first there was digital zoom, now theres complete digital camera repositioning!! XD
-
Can't wait to (hopefully) get my hands on this... I can easily incorporate it into my thrifty AGI robot concept / design. Currently looking to buy cheap camera modules from laptops which can be easily hacked and cheap.
-
Indeed its just off raw colours.
Sure you can have it ultron cause weve been talking, but this algorythm is copyright, except in certain circumstances. Means if I give it to you, your not allowed to give it to anyone, or I sue. No shit. :)
Took a long yonder in my head, and got the spirit back to get back typing, realizing the tracker is horrible and must be improved before any 3d calculation takes place.
its a big truck load into it, the 1kilobit name per pixel, I couldnt get it any faster than this, but the mesh is really solid, but it does still stuff up. I do have the 3d brute force type algorythm organized, but it may be a case of not going very fun speed, so far.
Once 3d is done, I then predict the back side of things, and do the proper segmentation, required to wrap around objects. With a bit of luck, itll be a pretty zany application.
http://www.youtube.com/watch?v=jjeL69827W0#ws (http://www.youtube.com/watch?v=jjeL69827W0#ws)
-
Fear not as I doubt I would ever understand your algorithm :P
The only thing I am interested in is combining some cool concepts and technologies such as your own to create a real working artificial mind - something you do not need billions of dollars and a huge computer cluster for, only a bit of brains (punny, isn't it?).
Anyway, would you mind discussing your idea on how your program would predict the 'invisible' side of an entity? I have had several hypothesis about how humans do this.
-
It didn't appear as if this 1024 one was as defined as your earlier 256 one. Then again, maybe it's me.
The music is um...interesting.
-
Impressive, I could recognise your face and hands quite well in that one.
-
I had a look at the code again, and I got it running faster. :) It always gets better the more I sit and tinker with something.
Ultron, I didnt believe I could get one of these video nets running on one gpu until very recently. The finished one will work on 4 or so at once, in sli.
Art, that music is my favorite for doing this mechanization stuff with. Futuristic drum and bass really puts me in the mood for making a robot. hehe. And trust me its definitely better with 1024 bit descriptor, I can vouch for that now I looked at it for ages, the surface positions arent stable without it.
How to see the backsides of things? Very simple, Just as a markov chain word predictor will predict words, you just have to think in 3d space instead, and wrap around the other side as if it was a sentence of words. Just saying generally.
So if you see a foot, you have a person.
If you see a basketball, maybe theres a kid.
If its done statistically and stochastically it may be quite amusing watching it come up with random scenes.
Bringing back animation, is as if Ive put all the capture into lots of backpropagator's, doing everything at once, and I bring it back like the animation in GTA V. But all done in a "greedy fell swooping fashion"... which is what my middle name phrase is, definitely not originality.
Which interestingly I dont think there is much originality required in the ai field, (especially thinking about computer vision.) its all a whole load of already done and thought about stuff that needs to be put together and implemented. Its just something you can do for yourself, so you dont have to pay anyone.
But not saying there isnt more to theorize, especially for less artificial vision, which this is very artificial - and has trouble with translucency.
I wonder how good some of the hollywood visual effects mocap programmers get their algorythms! I think it must be better than mine, some of the movies are pretty schmik. This 'universal capture' stuff has been out since the matrix movie with Keanu, but they did it in a camera room based system, and I guess these days its all about getting it work out of just one ordinary camera.
-
Seriously - a 4 nVidia SLI Graphic Card system? I thought making a custom machine with 2 cross-fire cards was expensive enough, but this?
-
Art- Just correcting myself, 256 actually is when it starts to get good, actually going past that only improves a little, but its worth it.
Just got to wait for the 980 to cost $400 then ill upgrade 2 more of them, and have 3 - in about 3 or so years.
Yeh I know im an nvidia idiot, remember back when intel cpus were so cool but so expensive. Im just an idiot for buying it, but they do go kinda smooth, but I know I wish I grew a brain and forgot about nvidia, I HATE CUDA, I only like direct compute anyway. CUDA takes directx shader code and makes it run ~8 times slower, I cant be bothered learning it, Its more for experts.
The track is nearly ready for me to go try out my 3d algorythm, but not quite yet.
One could think its specular which is ruining your track, but One could think its actually when two bodies are pulling apart. (which even happens on the nose, as it points out and rotates im thinking, also the side of the head, also the lips when they open.) The tracker doesnt know what of the bodies it belongs to. The same thing happens when one track's a 2d platformer, near the tile ground, it doesnt know to follow the tile or follow the character. And I think that is the main issue, not translucency/reflection.
https://www.youtube.com/watch?v=XCk0e-YvXXs&feature=youtu.be (https://www.youtube.com/watch?v=XCk0e-YvXXs&feature=youtu.be)
The obvious solution is I need a segmenter which makes sure it doesnt read the descriptor past the segment its centred apon. So thats what i'm doing next.
-
Hey ranch, let me know when you start running the final tests. I'd like to suggest a few algorithms or rather, concepts regarding artificial visions... I'l just let you know when they are mature enough.
-
Sure Ultra! :) But its going to take a bit more time, but its *nearly there* to getting the basic kinect 3d happening, ive got a whole kettle of fish to work out with the prediction memory. And then theres animation, but then the crazy inventions done.
trying to get a fill working in parallel is a bitch. This segmenter will then hone the tracker only on the important area, itll go quicker. This is only 1 step, ill post it when i get 12 steps going, and then itll look like little patches all over it, so the tracker can attend to more specific areas than the whole screen.
http://www.youtube.com/watch?v=kio4VfAjp_s#ws (http://www.youtube.com/watch?v=kio4VfAjp_s#ws)
here it is in action, i have a problem, i can only predict where to put the next cell to clip the tracking with, so it means using the old velocity and its definitely better this way but it always fails when something accellerates or decellerates.
http://youtu.be/FIjqUevTjBo (http://youtu.be/FIjqUevTjBo)
keeps your tesselation really solid with no memory, by guarding the points next tracks with the next predicted edges.
i keep postponing the 3d and classing cause i think ive got it then no, i have to go redo it again.
Its nothing unless the tracking is solid already, make sure you double check you have your reading pattern is bugless and spread out appropriately, i like 1024 bits at the moment.
-
Hey Ranch, sorry to drop into your topic without any particular cause, but I thought I would share this with the forum and specially you since (if I am not mistaken) relates to your project and I was hoping it might spark a few ideas in your mind... :)
http://princemio.net/portfolio/pathfinder/ (http://princemio.net/portfolio/pathfinder/)