[Freddy]

For fun I decided to hook Tyler up to an AIML engine.  Specifically Program#.

I was inspired by Bragi's use of that avatar system that I used to do the other Tyler videos.

I built my own avatar system using the same images I had used previously, except I did it in Visual Basic.

Then I hooked her up to the SAPI speech engine too.

All this was done in around 100 lines of code.



Those numbers you can see in the video are the viseme codes, or mouth shapes in other words.

I could only do a 30 second video as I am too tight to stump up the money to expand Fraps my screen capture software. 

Just a shame my spelling was so bad when rushed

[DaveMorton]

Well done, Freddy! That's a lot of content for only 100 lines of code! Keep up the good work!

[Data]

I like it Freddy, Tyler is coming to life in front of our eyes. Am also fond of the character, in a none sexual way I should add  .

Would of liked to see more, if you are quick you can capture video after video and patch them together in Virtual Dub, I used to do that until I got the full version of Fraps.

[Freddy]

I think I will stump up the money at some point, I think it was about $24 or something, so not a lot really for something that actually works right.

Thanks Dave, I think Program# allows you to do custom tags, but I only glanced at things as I was just wanting to see if the idea worked. With custom tags it should be possible to make a learning bot that scripts it's own responses.  I know SquareBear has done things like this, perhaps using the custom tags available at Pandorabots...I have yet to investigate properly.

But I think before I consider that it might be nice to hook her up to the ChatScript engine...hence my interest.  I will look at your project code for some pointers, for which I am very grateful indeed 

[DaveMorton]

That's why I'm here, Freddy. I can't describe the joy I feel, knowing that I was able to help another with some task or action. It's by far the most rewarding thing I can do.

[Freddy]

I like it Freddy, Tyler is coming to life in front of our eyes. Am also fond of the character, in a none sexual way I should add  .

Yes, she's of good character It shows that you don't have to always go for total photo realism to have a believable and engaging character.  I forget who made her, I should credit them in future, but yes they did a great job.

Dave, that's nice to hear

[Bragi]

yes, .net sapi engine is pretty easy to use. Getting basic visemes working is dead easy.
Nice work Freddy.

[Freddy]

Thanks, yes I found it quite easy really once my memory decided to come back. Easy when you know how I think. 

[Bragi]

things become trickier though, with voices of lesser quality, like some from cepstral. The version that you have of my engine doesn't handle these very well yet. I've got that fixed though. You did it in vb.net right? Wonder how well these voices do for you, try the 'robin' voice.

[Freddy]

Is Robin a free voice ?

You know another thing I was messing with was in Silverlight and using SAPI to make a wav file.  My theory (I know you will have already considered this) was you can have the voice on the server and just send wav audio to the browser. I can't see how to do the lip sync with a wav file though in Silverlight.

If it's possible it would be better; as the server could have a better quality voice than the one on the user's machine.  That was the trouble back in the day when you had a visually attractive avatar but when they spoke it was like they changed gender and put a sock in their mouth 

Trouble is, the server versions of decent voices cost $$$

[Freddy]

You did it in vb.net right?

Yes, but I think I will convert to C# as soon as possible.

[Bragi]

The only reason to convert to C# would be your own convenience. In the background all VS languages are run through the same compiler. The end result of a C# or VB.net (even a C++ app, with correct settings) is exactly the same. You don't get any 'language' advantages, only convenience. (C++ is a bit of an outsider as you can change it's settings and still go native or pure C with it)

The robin voice isn't free, but you can download a demo (which is what I did). It has an annoying 'buy me' message, but it's good enough for testing. Basically, the problem is that the 'viseme' event, doesn't happen at the time that it should 

Sending an mp3 over from a server to the client is a possibility. You would have to include the timing info of the visemes together with the mp3. But yes, I think you'll need a server voice for that, which is no doubt $$$. Another possibility would be to use the client's SAPI. The default MS voice isn't that good, but I think there is still room for improvement: you can use the 'say-as' ssml tag to let the voice know how to pronounce a word, which I think is the major issue: the default ms voice doesn't have a big default db.

[Freddy]

The only reason to convert to C# would be your own convenience.

Okay thanks

I'll look up the Robin voice.

Regarding the audio to browser route, there is also the ogg format and you can embed speech data into that as Hatpek does. Another possibility any way..

As for the $$$, Iovona (who made Tyler's voice) have a product called IVONA Speech Server but there is no price mentioned, it say's Contact Us which usually means it is more than you can afford.

[Bragi]

Regarding the audio to browser route, there is also the ogg format and you can embed speech data into that as Hatpek does. Another possibility any way..

Embedding timing info in the audio stream isn't that hard, in fact, that's how I fixed the 'issues' with Robin: it's a matter of collecting the time stamp and viseme index + sending it together with the data (this can be a simple text file).

[Bragi]

it say's Contact Us which usually means it is more than you can afford.

Yep + I really don't see why you need a 'full-standalone' server for this. It's easily made: you can stream the SAPI audio to a file, so no biggy.
(euhm that is, if the audio streaming also gives you some 'timing info', otherwise it's a different story (and clever one)