Flixz Avatar Project

  • 37 Replies
  • 13378 Views
*

Freddy

  • Administrator
  • **********************
  • Colossus
  • *
  • 6855
  • Mostly Harmless
Flixz Avatar Project
« on: February 06, 2013, 05:48:05 pm »
For the past few days I have been working on an avatar system for my projects.  So far I have done all this in vb.net.

For a long time I have wanted to get the visual quality of Daz/Poser renders into a real time animation.  This project is based on animation in real time, but the frames are pre-rendered.  In a way it's a little like the Verbots characters and the system Bragi uses for his project.  It was inspired by both.

So this little animation is really my prototype.  I wrote a few scripts to get Daz to render the required frames and model positions to save a lot of tiresome work.  I haven't yet used high quality renders as I want to quickly get the basic idea working.  So this animation is the basic Victoria 4 model without any enhancements.

The frame count is a modest 81 frames.  The idea here for me at first is to work on the 'idling' animation.  So that in real time she just moves about a bit, rather than a fixed frame looking out at you - instead her head should move about a little.

Enough rambling, remember, it's early days and it could be a lot smoother with more frames, but what do folks think so far ?

Animated Head Idle

*

Freddy

  • Administrator
  • **********************
  • Colossus
  • *
  • 6855
  • Mostly Harmless
Re: Flixz Avatar Project
« Reply #1 on: February 06, 2013, 05:50:49 pm »
BTW, the project name 'Flixz' comes from the word 'Flicks' as in the word for movies, but more in thought of when we were kids and drew little animations in the corners of books and flicked through them.  ;D

*

Bragi

  • Trusty Member
  • ********
  • Replicant
  • *
  • 564
    • Neural network design blog
Re: Flixz Avatar Project
« Reply #2 on: February 07, 2013, 07:20:29 am »
Very cool. Perhaps when it's working the way you like, you can integrate your library with my app?

*

Freddy

  • Administrator
  • **********************
  • Colossus
  • *
  • 6855
  • Mostly Harmless
Re: Flixz Avatar Project
« Reply #3 on: February 07, 2013, 11:05:01 am »
Thanks. :)  I'm still working on the idle algorithm to try and improve it, but it's interesting work.

And yes, I will eventually turn it into a dll/COM so it can be used in other projects.

I need to work on direct control next - moving from one place to another.  So for example a command to turn right, moves her head to the right following the frame path.

And I also need to do visemes.  Rather than render every single viseme for every position (81 frames x 20 visemes = 1620 extra frames) -  I think I will just have her talk in the forward facing position.  I think that is more natural too as people tend to look at you when they talk.  We will see.

*

Data

  • Trusty Member
  • ***********
  • Eve
  • *
  • 1279
  • Overclocked // Undervolted
    • Datahopa - Share your thoughts ideas and creations
Re: Flixz Avatar Project
« Reply #4 on: February 07, 2013, 11:23:38 am »
Only just spotted this.

Looking good, nicely done Freddy  O0

*

Bragi

  • Trusty Member
  • ********
  • Replicant
  • *
  • 564
    • Neural network design blog
Re: Flixz Avatar Project
« Reply #5 on: February 07, 2013, 12:43:33 pm »
Quote
Rather than render every single viseme for every position (81 frames x 20 visemes = 1620 extra frames) -  I think I will just have her talk in the forward facing position.
ye that's one of the problems you get with this technique. Of course, movies are also stored and played as partial images, so it should be doable. Keeping all the frames in memory is probably not a good idea.  You could group the viseme images per head position, and only load the ones that are currently being used. You'll probably best store all the images in a single file instead of separate images, cause searching for a file is a big cost, once it is open, reading a block can be pretty fast.
Also: crop all the images so that they are as small as possible. Saves lots of CPU time and disk space.

*

Freddy

  • Administrator
  • **********************
  • Colossus
  • *
  • 6855
  • Mostly Harmless
Re: Flixz Avatar Project
« Reply #6 on: February 07, 2013, 03:13:47 pm »
Yeah it's tricky.  So far I believe I have managed to store the 81 frames within memory - they are very small in size.  The impact on available memory is minimal.  Starting my program only takes up about 2-3% more memory.  Obviously it would be nice to have bigger images too.

At the moment I am trying to find ways to reduce CPU use, but as it is, it's not too bad.  With everything running so far I max out at about 30% CPU usage, and that includes the IDE of course.    All 81 images add up to only 3MB.

I am building in ways to minimize CPU use though - I already have a frames per second adjustment and a way to adjust how frequently the frames change.  So it should be possible to find a sweet spot.  Mostly I am working on 30fps at the moment and it seems okay.

This it never going to be a smooth and flexible as real-time 3d but I will get it as close as I can.

*

Freddy

  • Administrator
  • **********************
  • Colossus
  • *
  • 6855
  • Mostly Harmless
Re: Flixz Avatar Project
« Reply #7 on: February 07, 2013, 03:19:24 pm »
And thanks Data  :)

*

Bragi

  • Trusty Member
  • ********
  • Replicant
  • *
  • 564
    • Neural network design blog
Re: Flixz Avatar Project
« Reply #8 on: February 07, 2013, 03:56:33 pm »
Quote
This it never going to be a smooth and flexible as real-time 3d but I will get it as close as I can.
yep. That's true.

*

Freddy

  • Administrator
  • **********************
  • Colossus
  • *
  • 6855
  • Mostly Harmless
Re: Flixz Avatar Project
« Reply #9 on: February 13, 2013, 07:21:53 pm »
After a lot of fiddling I got a bit further, I am now using WPF which gives me DirectX, which results in less CPU use.  Thanks to Bragi for suggesting that :)

It's a small video, watch it on the higher resolution to see her better.

Now she is on the desktop.  I also invested in the new Genesis figure Victoria 5 from Daz, so the model is a lot nicer and with a much better skin.

The button is there so I can simulate a command(s).  For this demo she simply looks down and to the right , pauses and then moves to the top left, pause and then goes back to idling.

I still need to work on the eye movement.  I think I need to do some tweaks with the new models.  Not sure what to do with that aspect at the moment to be honest.

On to visemes for her next I think.

Desktop Avatar #2

*

Data

  • Trusty Member
  • ***********
  • Eve
  • *
  • 1279
  • Overclocked // Undervolted
    • Datahopa - Share your thoughts ideas and creations
Re: Flixz Avatar Project
« Reply #10 on: February 13, 2013, 07:57:25 pm »
There was a sign of disapproval when you removed the "new folder", a shake of the head, either she likes new folders or thinks you a little crazzy, or am I reading to much into this  :)

Cool Freddy  8)

*

Freddy

  • Administrator
  • **********************
  • Colossus
  • *
  • 6855
  • Mostly Harmless
Re: Flixz Avatar Project
« Reply #11 on: February 13, 2013, 08:03:29 pm »
Haha, well that's just coincidental unless its some kind of bizarre emergent thing ;)

Other than the two times I pressed the button, she pretty much looks anywhere she likes, within a defined area.  The button pushes her outside of that area so she can do more extreme movements.  I found after a while working on this, that you don't need a lot of idle head movement, but just enough to not be static.  My early trials had her moving her head a lot and she looked a bit mad, so I reduced that a bit.

Thanks for taking a look :)

 

*

Art

  • At the end of the game, the King and Pawn go into the same box.
  • Trusty Member
  • **********************
  • Colossus
  • *
  • 5865
Re: Flixz Avatar Project
« Reply #12 on: February 13, 2013, 11:37:25 pm »
I really liked that Freddy! Really cute character and a LOT more lifelike and believable than some others out there!

Good work so far!! O0
In the world of AI, it's the thought that counts!

*

Sybershot

  • Trusty Member
  • **
  • Bumblebee
  • *
  • 42
    • Scripted Intelligence
Re: Flixz Avatar Project
« Reply #13 on: February 14, 2013, 12:18:27 am »
Great work there Freddy, She looks fabulous. Keep up the great work :)

*

Bragi

  • Trusty Member
  • ********
  • Replicant
  • *
  • 564
    • Neural network design blog
Re: Flixz Avatar Project
« Reply #14 on: February 14, 2013, 07:21:24 am »
yep, starting to come along nicely.

 


OpenAI Speech-to-Speech Reasoning Demo
by ivan.moony (AI News )
Today at 01:31:53 pm
Say good-bye to GPUs...
by MikeB (AI News )
March 23, 2024, 09:23:52 am
Google Bard report
by ivan.moony (AI News )
February 14, 2024, 04:42:23 pm
Elon Musk's xAI Grok Chatbot
by MikeB (AI News )
December 11, 2023, 06:26:33 am
Nvidia Hype
by 8pla.net (AI News )
December 06, 2023, 10:04:52 pm
How will the OpenAI CEO being Fired affect ChatGPT?
by 8pla.net (AI News )
December 06, 2023, 09:54:25 pm
Independent AI sovereignties
by WriterOfMinds (AI News )
November 08, 2023, 04:51:21 am
LLaMA2 Meta's chatbot released
by 8pla.net (AI News )
October 18, 2023, 11:41:21 pm

Users Online

267 Guests, 0 Users

Most Online Today: 343. Most Online Ever: 2369 (November 21, 2020, 04:08:13 pm)

Articles