Ai Dreams Forum

Chatbots => Avatar Talk => Topic started by: bot_animator on July 28, 2017, 10:11:21 pm

Title: New javascript library for animating Chatbots on the web - talkr
Post by: bot_animator on July 28, 2017, 10:11:21 pm
[ 6/4/2018 update - check out the end of this thread for an improved replacement library: https://github.com/talkr-app/talkr-apng]

Hi Everyone,

I wanted to introduce a new javascript library ( https://github.com/talkr-app/gif-talkr  (https://github.com/talkr-app/gif-talkr)) that may be useful for animating text-to-speech enabled Chatbots on the web.  Here is a demo that will play an arbitrary TTS message back to you.  ( http://talkrapp.com/gifdemo/example.html (http://talkrapp.com/gifdemo/example.html))

The technique only requires a GIF file, but I hope you are pleasantly surprised by the quality of animations you can achieve.  You can see a library of sample GIF files here (http://imgur.com/a/NRZVQ (http://imgur.com/a/NRZVQ)) .  You could in theory use any GIF file of a talking character (see the github page for details), but ideally you can use a GIF file with a high quality animation and a special GIF extension that enables independent blink and eyebrow movements.  A future version of our iOS app talkr (http://talkrapp.com (http://talkrapp.com)) will be able to export these GIF files.  As of now, you can only export movies from talkr which you would have to convert to a normal GIF without independent blink and eyebrow channels.

In any case, I welcome any feedback on the library or the app!  There should be nothing stopping you from bringing your chatbots to any platform that supports TTS in the browser (iOS, Android, Windows, OSX, etc)






Title: Re: New javascript library for animating Chatbots on the web - talkr
Post by: Freddy on July 29, 2017, 12:02:24 am
Hi there,

This is a nice system and I look forward to seeing the app being able to export the GIFs that botmasters will need.

In the meantime the example GIFs are still very usable as a bot avatar : http://elfscript.uk/chattygirl.html

What I found really cool was how convincing you made her without having to play with all that phoneme/viseme stuff.

I took the liberty of giving you a forum avatar :)
Title: Re: New javascript library for animating Chatbots on the web - talkr
Post by: 8pla.net on July 29, 2017, 01:26:52 am
Freddy,

This library works with any GIF animation image.  A gifanim talking (mouth moving) and eyes blinking works best.

New Sample:  http://chatbot.altervista.org/giftts/moretesting.php

However for example, a gifanim of people dancing would vary following different user inputs.

(https://media.giphy.com/media/h1ojsprlDCgbS/giphy.gif)

Image Citation: Giphy.com



Title: Re: New javascript library for animating Chatbots on the web - talkr
Post by: Freddy on July 29, 2017, 02:12:02 am
I just see a static image there, no animation in Chrome.

She really thinks she can dance doesn't she  ;D
Title: Re: New javascript library for animating Chatbots on the web - talkr
Post by: bot_animator on July 29, 2017, 02:21:02 am
I get the animation when I hit "Chat" on chrome on OSX.   That Rapunzel gif is a good one 8pla.net!  Also, that javascript TTS is a bit faster than the ones I've been using...I would definitely decrease the per-character estimate a tad (line 157 of example.js) from 50 to maybe 40 or so to get a better sync with that TTS. 
Title: Re: New javascript library for animating Chatbots on the web - talkr
Post by: Freddy on July 29, 2017, 02:43:42 am
Hmm tried it in FF and it did nothing either. Edge really didn't like it again. I think Chrome is up to date here, it should be.
Title: Re: New javascript library for animating Chatbots on the web - talkr
Post by: Freddy on July 29, 2017, 02:45:53 am
It was a little out of date, but after updating I just see a static image again.

Version 60.0.3112.78 (Official Build) (64-bit)
Title: Re: New javascript library for animating Chatbots on the web - talkr
Post by: 8pla.net on July 29, 2017, 03:23:08 am
I just got it working under Chrome too now.

i removed some PHP I added for alpha testing to stop page caching.

Code
?r=<?=rand(111,999)?>





Title: Re: New javascript library for animating Chatbots on the web - talkr
Post by: Freddy on July 29, 2017, 03:43:25 am
That's got it  O0

Working fine now, really good choice of character, she looks crazy !
Title: Re: New javascript library for animating Chatbots on the web - talkr
Post by: 8pla.net on July 29, 2017, 03:52:11 am
I would definitely decrease the per-character estimate a tad (line 157 of example.js) from 50 to maybe 40 or so to get a better sync with that TTS.

Thank you for your precious feedback.  I decreased the per-character estimate to luck number 38.
Title: Re: New javascript library for animating Chatbots on the web - talkr
Post by: 8pla.net on July 29, 2017, 03:58:23 am
That's got it  O0

Working fine now, really good choice of character, she looks crazy !

Thank you Freddy.  It easily worked with several other GIF anims.  And when
it did not work for visemes (lip postures), the results were still fun to watch.
My suggestion is to try it with any of the many GIF anims you can find online.
This library actually works great with this little guy, (talking and blinking)...  :tickedoff:
Title: Re: New javascript library for animating Chatbots on the web - talkr
Post by: bot_animator on August 22, 2017, 10:37:05 pm


Just a quick update that I added blink/eyebrow idle options.  Also hooked it up to a API.AI here (my first experience with bot programming): https://talkrapp.com/gifdemo/bot/index.html (https://talkrapp.com/gifdemo/bot/index.html)

The demo works best on Windows/MAC using Chrome so you can use the speech recognition.  It also works on Android, but I could only get one English voice to work on my samsung galaxy tab.  (If anyone has an Android, especially non-samsung to try, I'd be interested to know what happens when you say "change voice".  Does the english voice change?)

Try these commands:

"Change avatar"
"Learn new response"
"Ask me about the presidents"
"Give me a word problem"
"Yes or no, will I become president one day?"




Title: Re: New javascript library for animating Chatbots on the web - talkr
Post by: Freddy on August 23, 2017, 02:47:18 am
Thanks for the update  :)

I got a message that you had updated it, but have not had a chance top play with it yet. I'll try to update the page I have with the new version.

Thank you  O0
Title: Re: New javascript library for animating Chatbots on the web - talkr
Post by: Freddy on September 27, 2017, 11:27:26 pm
I had a play with this and whilst she blinks during idle time (which is great) I can't get the eyebrows to do the same. I noticed your link to her also does not have her eyebrows moving idly - so is that done yet and should I stop pulling my hair out now ?

;D ;)

This is in Chrome btw.
Title: Re: New javascript library for animating Chatbots on the web - talkr
Post by: Freddy on September 28, 2017, 08:06:57 pm
With some further experiments I found that it would be useful to resize the avatar in the script somehow.

I was able to reduce the size of the avatar by hacking in some CSS onto the gif and the canvas, but this resulted in glitchy animation.

Of course I could probably solve this by using a gif editor, but I thought I would mention it.

Edit : yes this did the trick : https://ezgif.com

It's all very nice work though :)
Title: Re: New javascript library for animating Chatbots on the web - talkr
Post by: bot_animator on October 05, 2017, 05:37:22 pm
By default, I set up the blinks to trigger randomly even when the avatar is not talking.  I set up the eyebrows to trigger randomly but only during speech.  Could that explain the no eyebrows?  These settings are configurable from javascript, but that is how it is set up by default. 

I'm looking into what could cause choppy animation with CSS and see if there is anything I can do about it.
Title: Re: New javascript library for animating Chatbots on the web - talkr
Post by: Freddy on October 12, 2017, 12:51:42 am
Thanks, yes that explains it  O0
Title: Re: New javascript library for animating Chatbots on the web - talkr
Post by: bot_animator on November 10, 2017, 12:45:46 pm

Just a quick note, that the new version of talkr that supports GIF export has been released, so you can make your own GIF avatars from any picture.  You need to purchase the "Premium" version ($2.99) and enable the GIF export in the settings.

www.talkrapp.com
Title: Re: New javascript library for animating Chatbots on the web - talkr
Post by: 8pla.net on November 10, 2017, 01:20:40 pm
Congratulations on launching a killer app!
Title: Re: New javascript library for animating Chatbots on the web - talkr
Post by: bot_animator on June 04, 2018, 10:44:32 pm
Just a quick note that I have deprecated the gif-talkr project in favor of the new talkr-apng (https://github.com/talkr-app/talkr-apng (https://github.com/talkr-app/talkr-apng)) library.  There are huge improvements in memory usage and loading times.  And the PNG format fully supports transparency so you could have an independent background.  I also created the talkr-tts (https://github.com/talkr-app/talkr-tts (https://github.com/talkr-app/talkr-tts)) library to facilitate playing back tts clips and scenes.

For a demo of both libraries, check out talkrapp.com/apngdemo/index.html (http://talkrapp.com/apngdemo/index.html)