[8pla.net]

Here is a (temporary) video of a test run I just did with my Lip Sync hobby
project now being written in PHP.   Warning: This is unfinished work.   

https://www.youtube.com/v/E2BMywBxjbE&rel=0&info=0

It uses the GD and Image Functions to draw the avatar in PHP and the
Festival speech synthesizer.

[Freddy]

Nice  How many visemes are you using at the moment ?

I have a 2D project too here : http://aidreams.co.uk/forum/index.php?topic=5045

That's all in Windows at the moment, but I have had thoughts about converting it to the web.

Festival is good, have you tried MaryTTS yet ? It's very nice too...

http://lnv-90208.sb.dfki.de/

[8pla.net]

Freddy asked, "Nice  How many visemes are you using at the moment ?"

Oh, I don't even have a final viseme set yet to be honest.  I am still trying to reduce the wobbliness in the visemes.  So, that count goes up and down depending on which frames per second combination animates the smoothest.

Freddy said, "I have a 2D project too here : http://aidreams.co.uk/forum/index.php?topic=5045"

That is a very cool 2D project you got there.    May I suggest removing the meshes for the body, leaving everything from the neck up?

Freddy said, "That's all in Windows at the moment, but I have had thoughts about converting it to the web."

If you don't mind another suggestion, perhaps you may get started with testing on a localhost running on a Microsoft IIS webserver, and then make a demo video of it running to show its potential.

Freddy asked, "Festival is good, have you tried MaryTTS yet ? It's very nice too..."

Indeed, I have briefly. Yet, I like to have the project looking like a work in progress during early testing.

Here is an update to this work in progress: https://www.youtube.com/v/vEFk94LAjQ8?rel=0&info=0

[Art]

No offense intended but all I can see are the lips pretty much constantly moving without any pauses between words.

Does or will it pause the lip movement if speech is slowed down or is that not an option at present?

Please keep us posted in this interesting approach.

[8pla.net]

Please let me clarify that the binary black and white 2D design process is the beginning step.  And the blue hair and red mouth are subsequent steps in 3D glasses support.   

Art, no offense taken at all.  On the contrary, thank you for actually helping me with your valuable feedback.  Before we continue our discussion, please let me check with a moderator that 3D glasses are considered on topic for this thread.

[Freddy]

That is a very cool 2D project you got there.    May I suggest removing the meshes for the body, leaving everything from the neck up?

Thanks

If I were to move this to a web based animation then I probably would do that yes. It's just when I am developing something at first I may take a few short cuts - like not removing the body in this case.

If you don't mind another suggestion, perhaps you may get started with testing on a localhost running on a Microsoft IIS webserver, and then make a demo video of it running to show its potential.

I don't mind suggestions. I'm more at home with Apache to be honest I use it a fair bit with web stuff. Recently I built a Linux box and put a server on it that will some time in the future be used for various things. One of them as my voice server (MaryTTS) as well as any animated characters I might demonstrate.

Here is an update to this work in progress...

She looks cute Did you find a way to time visemes firing in Festival ?

...please let me check with a moderator that 3D glasses are considered on topic for this thread.

That's fine by me. Art is a moderator as well. We are not too strict about it here. If something goes way off at a tangent we can just spilt the topic off.

[8pla.net]

Art asked, "Does or will it pause the lip movement if speech is slowed down or is that not an option at present?"

Freddy mentioned, "She looks cute Did you find a way to time visemes firing in Festival ?"

Bingo! The wobbliness in the lip movement is due to the frames per second (fps) being too fast.  There may be a noticeable improvement here...  https://www.youtube.com/v/jH80QJr3ddg?rel=0&info=0  Yet, it is still unfinished, even for a trial run.  Now, believe it or not, the optimizations I attempted on the visemes may no longer be needed, so I may have to undo them.   So, the viseme count may be going back up again.

The reason I chose this funny poem for her to read was that it is supposed to contain every phoneme sound.

[Art]

Agreed, coming along with improvements. I had my 3D glasses on too but I really don't get much out of those (just mho).

[8pla.net]

Thanks, Art, for testing with your 3D glasses.  I really appreciate that very much!  I still have to order my pair of 3D (red/blue) glasses. So this time I had to guess the two different perspectives in this experiment.  So, I am still doing that math now. However, the first part of the experiment was rendering the color palette, which I think was at least a partial success.  The support of 3D glasses is still in very early stages of development.

[Art]

I also have a pair with clear lenses that operate on an electronic shutter principle where each eyeglass lens opens and closes alternating with Left, Right and so on at a rather fast, almost undetectable rate.
It was used on a variety of 3D projects, games, VR, etc., several years ago. Mine works on tiny batteries instead of being plugged in.

Nice concept but honestly, we live in a 3D world but aside from optical correction like myopia or astigmatism those 3D glasses are really nothing more than a curiosity for the general population instead of a really useful, everyday item. Just MHO and please, don't sway your project if my comments seem to be raining on your parade. I mean no harm, only a general observation from my lifetime. I just don't think people want to have to wear a pair of glasses to see 3D like they did in the theaters in the 1950's and 60's.

Then there's those wonderfully stylish and ever accessible, Apple iGlass(es). Go figure! Maybe Brin will come up with a different type of Google Goggles! (instead of the app).

Good luck with the rest of your project!

[8pla.net]

@Freddy: 

I was curious as to whether you may be working with some type of alpha channel for a partial transparency on the hair for your model?

Also there appears to be an audio skew correction now.  Perhaps it was You Tube doing maintenance or simply my Internet connection before?

[Freddy]

The tool kit I was using (Helix 3D Toolkit for WPF) has limited support for transparency. At the moment or at least in the last few months I was only able to use transparent PNG textures.

As you can see in the video it does not render that well where strands of hair overlap. Although the eyelashes are fine, I think mainly because there is nothing else in front of them.

I tried a few things, like depth sorting the polys but to be honest I was out of my own depth there and I could not get anything to work.

Apart from that Helix tool kit is a great bit of kit and is one of the easier ways into 3D on Windows and .NET that I have found. You can do a lot with it and it helped me learn more about how 3D works.

[Freddy]

Also there appears to be an audio skew correction now.  Perhaps it was You Tube doing maintenance or simply my Internet connection before?

No idea sorry - I can say I have not changed anything though.

[8pla.net]

@Art:

Well, I just ordered brand new (never used) 3D glasses from Ebay for 99 cents with free shipping from Hong Kong.  They are plastic and look like Clark Kent Superman style glasses, except they have one red lens and one blue lens.  If this experiment is successful, then I plan to make 3D glasses optional using a radio button on the website.

I am also researching the other 3D glasses options for the long term.  From what I have read so far, it seems advanced 3D glasses, such as those you mentioned, may require specialized theater equipment.

[8pla.net]

@Freddy,
I have to review the technical terms, but in simple terms, I remember there was a pair of inverted black and white monochrome images that are the exact opposite of each other used in this rendering.  The point being that they may not be modified separately since they have to match black to white and vice versa.   So if you modify one, then make a copy, invert it and regenerate the second one.

I am not sure if this applies to your workflow.  I am just brainstorming.  If this sounds familiar then I will research it in more detail.  In any case, I like this idea, and I think the results may turn out to be efficient and realistic.