Tyler AIML

  • 15 Replies
  • 6394 Views
*

Freddy

  • Administrator
  • **********************
  • Colossus
  • *
  • 6855
  • Mostly Harmless
Tyler AIML
« on: July 15, 2011, 05:08:01 pm »
For fun I decided to hook Tyler up to an AIML engine.  Specifically Program#.

I was inspired by Bragi's use of that avatar system that I used to do the other Tyler videos.

I built my own avatar system using the same images I had used previously, except I did it in Visual Basic.

Then I hooked her up to the SAPI speech engine too.

All this was done in around 100 lines of code.

http://youtu.be/Ree9NGWikB8

Those numbers you can see in the video are the viseme codes, or mouth shapes in other words.

I could only do a 30 second video as I am too tight to stump up the money to expand Fraps my screen capture software.  ;D

Just a shame my spelling was so bad when rushed ;)

*

DaveMorton

  • Trusty Member
  • ********
  • Replicant
  • *
  • 636
  • Safe, Reliable Insanity, Since 1961
    • Geek Cave Creations
Re: Tyler AIML
« Reply #1 on: July 15, 2011, 05:22:30 pm »
Well done, Freddy! That's a lot of content for only 100 lines of code! Keep up the good work! :D
Comforting the Disturbed, Disturbing the Comfortable
Chat with Morti!
LinkedIn Profile
CAPTCHA4us

*

Data

  • Trusty Member
  • ***********
  • Eve
  • *
  • 1279
  • Overclocked // Undervolted
    • Datahopa - Share your thoughts ideas and creations
Re: Tyler AIML
« Reply #2 on: July 15, 2011, 05:23:03 pm »
I like it Freddy, Tyler is coming to life in front of our eyes. Am also fond of the character, in a none sexual way I should add  :).

Would of liked to see more, if you are quick you can capture video after video and patch them together in Virtual Dub, I used to do that until I got the full version of Fraps.

*

Freddy

  • Administrator
  • **********************
  • Colossus
  • *
  • 6855
  • Mostly Harmless
Re: Tyler AIML
« Reply #3 on: July 15, 2011, 05:32:44 pm »
I think I will stump up the money at some point, I think it was about $24 or something, so not a lot really for something that actually works right.

Thanks Dave, I think Program# allows you to do custom tags, but I only glanced at things as I was just wanting to see if the idea worked. With custom tags it should be possible to make a learning bot that scripts it's own responses.  I know SquareBear has done things like this, perhaps using the custom tags available at Pandorabots...I have yet to investigate properly.

But I think before I consider that it might be nice to hook her up to the ChatScript engine...hence my interest.  I will look at your project code for some pointers, for which I am very grateful indeed  O0
« Last Edit: July 15, 2011, 06:11:00 pm by Freddy »

*

DaveMorton

  • Trusty Member
  • ********
  • Replicant
  • *
  • 636
  • Safe, Reliable Insanity, Since 1961
    • Geek Cave Creations
Re: Tyler AIML
« Reply #4 on: July 15, 2011, 05:35:47 pm »
That's why I'm here, Freddy. I can't describe the joy I feel, knowing that I was able to help another with some task or action. It's by far the most rewarding thing I can do.
Comforting the Disturbed, Disturbing the Comfortable
Chat with Morti!
LinkedIn Profile
CAPTCHA4us

*

Freddy

  • Administrator
  • **********************
  • Colossus
  • *
  • 6855
  • Mostly Harmless
Re: Tyler AIML
« Reply #5 on: July 15, 2011, 05:39:11 pm »
I like it Freddy, Tyler is coming to life in front of our eyes. Am also fond of the character, in a none sexual way I should add  :).
Yes, she's of good character :) It shows that you don't have to always go for total photo realism to have a believable and engaging character.  I forget who made her, I should credit them in future, but yes they did a great job.

Dave, that's nice to hear :)

*

Bragi

  • Trusty Member
  • ********
  • Replicant
  • *
  • 564
    • Neural network design blog
Re: Tyler AIML
« Reply #6 on: July 16, 2011, 09:04:24 am »
yes, .net sapi engine is pretty easy to use. Getting basic visemes working is dead easy.
Nice work Freddy.

*

Freddy

  • Administrator
  • **********************
  • Colossus
  • *
  • 6855
  • Mostly Harmless
Re: Tyler AIML
« Reply #7 on: July 17, 2011, 12:39:30 pm »
Thanks, yes I found it quite easy really once my memory decided to come back. Easy when you know how I think.  ;)

*

Bragi

  • Trusty Member
  • ********
  • Replicant
  • *
  • 564
    • Neural network design blog
Re: Tyler AIML
« Reply #8 on: July 17, 2011, 12:57:30 pm »
things become trickier though, with voices of lesser quality, like some from cepstral. The version that you have of my engine doesn't handle these very well yet. I've got that fixed though. You did it in vb.net right? Wonder how well these voices do for you, try the 'robin' voice.

*

Freddy

  • Administrator
  • **********************
  • Colossus
  • *
  • 6855
  • Mostly Harmless
Re: Tyler AIML
« Reply #9 on: July 17, 2011, 01:22:34 pm »
Is Robin a free voice ?

You know another thing I was messing with was in Silverlight and using SAPI to make a wav file.  My theory (I know you will have already considered this) was you can have the voice on the server and just send wav audio to the browser. I can't see how to do the lip sync with a wav file though in Silverlight.

If it's possible it would be better; as the server could have a better quality voice than the one on the user's machine.  That was the trouble back in the day when you had a visually attractive avatar but when they spoke it was like they changed gender and put a sock in their mouth  ;D

Trouble is, the server versions of decent voices cost $$$

*

Freddy

  • Administrator
  • **********************
  • Colossus
  • *
  • 6855
  • Mostly Harmless
Re: Tyler AIML
« Reply #10 on: July 17, 2011, 01:23:22 pm »
You did it in vb.net right?
Yes, but I think I will convert to C# as soon as possible.

*

Bragi

  • Trusty Member
  • ********
  • Replicant
  • *
  • 564
    • Neural network design blog
Re: Tyler AIML
« Reply #11 on: July 17, 2011, 02:18:45 pm »
The only reason to convert to C# would be your own convenience. In the background all VS languages are run through the same compiler. The end result of a C# or VB.net (even a C++ app, with correct settings) is exactly the same. You don't get any 'language' advantages, only convenience. (C++ is a bit of an outsider as you can change it's settings and still go native or pure C with it)

The robin voice isn't free, but you can download a demo (which is what I did). It has an annoying 'buy me' message, but it's good enough for testing. Basically, the problem is that the 'viseme' event, doesn't happen at the time that it should  :-\

Sending an mp3 over from a server to the client is a possibility. You would have to include the timing info of the visemes together with the mp3. But yes, I think you'll need a server voice for that, which is no doubt $$$. Another possibility would be to use the client's SAPI. The default MS voice isn't that good, but I think there is still room for improvement: you can use the 'say-as' ssml tag to let the voice know how to pronounce a word, which I think is the major issue: the default ms voice doesn't have a big default db.

*

Freddy

  • Administrator
  • **********************
  • Colossus
  • *
  • 6855
  • Mostly Harmless
Re: Tyler AIML
« Reply #12 on: July 17, 2011, 02:26:43 pm »
Quote
The only reason to convert to C# would be your own convenience.

Okay thanks :)

I'll look up the Robin voice.

Regarding the audio to browser route, there is also the ogg format and you can embed speech data into that as Hatpek does. Another possibility any way..

As for the $$$, Iovona (who made Tyler's voice) have a product called IVONA Speech Server but there is no price mentioned, it say's Contact Us which usually means it is more than you can afford.

*

Bragi

  • Trusty Member
  • ********
  • Replicant
  • *
  • 564
    • Neural network design blog
Re: Tyler AIML
« Reply #13 on: July 17, 2011, 02:46:53 pm »
Quote
Regarding the audio to browser route, there is also the ogg format and you can embed speech data into that as Hatpek does. Another possibility any way..
Embedding timing info in the audio stream isn't that hard, in fact, that's how I fixed the 'issues' with Robin: it's a matter of collecting the time stamp and viseme index + sending it together with the data (this can be a simple text file).

*

Bragi

  • Trusty Member
  • ********
  • Replicant
  • *
  • 564
    • Neural network design blog
Re: Tyler AIML
« Reply #14 on: July 17, 2011, 02:52:56 pm »
Quote
it say's Contact Us which usually means it is more than you can afford.
Yep + I really don't see why you need a 'full-standalone' server for this. It's easily made: you can stream the SAPI audio to a file, so no biggy.
(euhm that is, if the audio streaming also gives you some 'timing info', otherwise it's a different story (and clever one)

 


OpenAI Speech-to-Speech Reasoning Demo
by ivan.moony (AI News )
March 28, 2024, 01:31:53 pm
Say good-bye to GPUs...
by MikeB (AI News )
March 23, 2024, 09:23:52 am
Google Bard report
by ivan.moony (AI News )
February 14, 2024, 04:42:23 pm
Elon Musk's xAI Grok Chatbot
by MikeB (AI News )
December 11, 2023, 06:26:33 am
Nvidia Hype
by 8pla.net (AI News )
December 06, 2023, 10:04:52 pm
How will the OpenAI CEO being Fired affect ChatGPT?
by 8pla.net (AI News )
December 06, 2023, 09:54:25 pm
Independent AI sovereignties
by WriterOfMinds (AI News )
November 08, 2023, 04:51:21 am
LLaMA2 Meta's chatbot released
by 8pla.net (AI News )
October 18, 2023, 11:41:21 pm

Users Online

294 Guests, 0 Users

Most Online Today: 363. Most Online Ever: 2369 (November 21, 2020, 04:08:13 pm)

Articles