New javascript library for animating Chatbots on the web - talkr

  • 19 Replies
  • 12882 Views
*

bot_animator

  • Trusty Member
  • *
  • Roomba
  • *
  • 6
[ 6/4/2018 update - check out the end of this thread for an improved replacement library: https://github.com/talkr-app/talkr-apng]

Hi Everyone,

I wanted to introduce a new javascript library ( https://github.com/talkr-app/gif-talkr ) that may be useful for animating text-to-speech enabled Chatbots on the web.  Here is a demo that will play an arbitrary TTS message back to you.  ( http://talkrapp.com/gifdemo/example.html)

The technique only requires a GIF file, but I hope you are pleasantly surprised by the quality of animations you can achieve.  You can see a library of sample GIF files here (http://imgur.com/a/NRZVQ) .  You could in theory use any GIF file of a talking character (see the github page for details), but ideally you can use a GIF file with a high quality animation and a special GIF extension that enables independent blink and eyebrow movements.  A future version of our iOS app talkr (http://talkrapp.com) will be able to export these GIF files.  As of now, you can only export movies from talkr which you would have to convert to a normal GIF without independent blink and eyebrow channels.

In any case, I welcome any feedback on the library or the app!  There should be nothing stopping you from bringing your chatbots to any platform that supports TTS in the browser (iOS, Android, Windows, OSX, etc)






« Last Edit: June 04, 2018, 10:46:53 pm by bot_animator »

*

Freddy

  • Administrator
  • **********************
  • Colossus
  • *
  • 6860
  • Mostly Harmless
Re: New javascript library for animating Chatbots on the web - talkr
« Reply #1 on: July 29, 2017, 12:02:24 am »
Hi there,

This is a nice system and I look forward to seeing the app being able to export the GIFs that botmasters will need.

In the meantime the example GIFs are still very usable as a bot avatar : http://elfscript.uk/chattygirl.html

What I found really cool was how convincing you made her without having to play with all that phoneme/viseme stuff.

I took the liberty of giving you a forum avatar :)

*

8pla.net

  • Trusty Member
  • ***********
  • Eve
  • *
  • 1307
  • TV News. Pub. UAL (PhD). Robitron Mod. LPC Judge.
    • 8pla.net
Re: New javascript library for animating Chatbots on the web - talkr
« Reply #2 on: July 29, 2017, 01:26:52 am »
Freddy,

This library works with any GIF animation image.  A gifanim talking (mouth moving) and eyes blinking works best.

New Sample:  http://chatbot.altervista.org/giftts/moretesting.php

However for example, a gifanim of people dancing would vary following different user inputs.



Image Citation: Giphy.com



My Very Enormous Monster Just Stopped Using Nine

*

Freddy

  • Administrator
  • **********************
  • Colossus
  • *
  • 6860
  • Mostly Harmless
Re: New javascript library for animating Chatbots on the web - talkr
« Reply #3 on: July 29, 2017, 02:12:02 am »
I just see a static image there, no animation in Chrome.

She really thinks she can dance doesn't she  ;D

*

bot_animator

  • Trusty Member
  • *
  • Roomba
  • *
  • 6
Re: New javascript library for animating Chatbots on the web - talkr
« Reply #4 on: July 29, 2017, 02:21:02 am »
I get the animation when I hit "Chat" on chrome on OSX.   That Rapunzel gif is a good one 8pla.net!  Also, that javascript TTS is a bit faster than the ones I've been using...I would definitely decrease the per-character estimate a tad (line 157 of example.js) from 50 to maybe 40 or so to get a better sync with that TTS. 

*

Freddy

  • Administrator
  • **********************
  • Colossus
  • *
  • 6860
  • Mostly Harmless
Re: New javascript library for animating Chatbots on the web - talkr
« Reply #5 on: July 29, 2017, 02:43:42 am »
Hmm tried it in FF and it did nothing either. Edge really didn't like it again. I think Chrome is up to date here, it should be.

*

Freddy

  • Administrator
  • **********************
  • Colossus
  • *
  • 6860
  • Mostly Harmless
Re: New javascript library for animating Chatbots on the web - talkr
« Reply #6 on: July 29, 2017, 02:45:53 am »
It was a little out of date, but after updating I just see a static image again.

Version 60.0.3112.78 (Official Build) (64-bit)

*

8pla.net

  • Trusty Member
  • ***********
  • Eve
  • *
  • 1307
  • TV News. Pub. UAL (PhD). Robitron Mod. LPC Judge.
    • 8pla.net
Re: New javascript library for animating Chatbots on the web - talkr
« Reply #7 on: July 29, 2017, 03:23:08 am »
I just got it working under Chrome too now.

i removed some PHP I added for alpha testing to stop page caching.

Code
?r=<?=rand(111,999)?>





My Very Enormous Monster Just Stopped Using Nine

*

Freddy

  • Administrator
  • **********************
  • Colossus
  • *
  • 6860
  • Mostly Harmless
Re: New javascript library for animating Chatbots on the web - talkr
« Reply #8 on: July 29, 2017, 03:43:25 am »
That's got it  O0

Working fine now, really good choice of character, she looks crazy !

*

8pla.net

  • Trusty Member
  • ***********
  • Eve
  • *
  • 1307
  • TV News. Pub. UAL (PhD). Robitron Mod. LPC Judge.
    • 8pla.net
Re: New javascript library for animating Chatbots on the web - talkr
« Reply #9 on: July 29, 2017, 03:52:11 am »
I would definitely decrease the per-character estimate a tad (line 157 of example.js) from 50 to maybe 40 or so to get a better sync with that TTS.

Thank you for your precious feedback.  I decreased the per-character estimate to luck number 38.
My Very Enormous Monster Just Stopped Using Nine

*

8pla.net

  • Trusty Member
  • ***********
  • Eve
  • *
  • 1307
  • TV News. Pub. UAL (PhD). Robitron Mod. LPC Judge.
    • 8pla.net
Re: New javascript library for animating Chatbots on the web - talkr
« Reply #10 on: July 29, 2017, 03:58:23 am »
That's got it  O0

Working fine now, really good choice of character, she looks crazy !

Thank you Freddy.  It easily worked with several other GIF anims.  And when
it did not work for visemes (lip postures), the results were still fun to watch.
My suggestion is to try it with any of the many GIF anims you can find online.
This library actually works great with this little guy, (talking and blinking)...  :tickedoff:
My Very Enormous Monster Just Stopped Using Nine

*

bot_animator

  • Trusty Member
  • *
  • Roomba
  • *
  • 6
Re: New javascript library for animating Chatbots on the web - talkr
« Reply #11 on: August 22, 2017, 10:37:05 pm »


Just a quick update that I added blink/eyebrow idle options.  Also hooked it up to a API.AI here (my first experience with bot programming): https://talkrapp.com/gifdemo/bot/index.html

The demo works best on Windows/MAC using Chrome so you can use the speech recognition.  It also works on Android, but I could only get one English voice to work on my samsung galaxy tab.  (If anyone has an Android, especially non-samsung to try, I'd be interested to know what happens when you say "change voice".  Does the english voice change?)

Try these commands:

"Change avatar"
"Learn new response"
"Ask me about the presidents"
"Give me a word problem"
"Yes or no, will I become president one day?"





*

Freddy

  • Administrator
  • **********************
  • Colossus
  • *
  • 6860
  • Mostly Harmless
Re: New javascript library for animating Chatbots on the web - talkr
« Reply #12 on: August 23, 2017, 02:47:18 am »
Thanks for the update  :)

I got a message that you had updated it, but have not had a chance top play with it yet. I'll try to update the page I have with the new version.

Thank you  O0

*

Freddy

  • Administrator
  • **********************
  • Colossus
  • *
  • 6860
  • Mostly Harmless
Re: New javascript library for animating Chatbots on the web - talkr
« Reply #13 on: September 27, 2017, 11:27:26 pm »
I had a play with this and whilst she blinks during idle time (which is great) I can't get the eyebrows to do the same. I noticed your link to her also does not have her eyebrows moving idly - so is that done yet and should I stop pulling my hair out now ?

;D ;)

This is in Chrome btw.

*

Freddy

  • Administrator
  • **********************
  • Colossus
  • *
  • 6860
  • Mostly Harmless
Re: New javascript library for animating Chatbots on the web - talkr
« Reply #14 on: September 28, 2017, 08:06:57 pm »
With some further experiments I found that it would be useful to resize the avatar in the script somehow.

I was able to reduce the size of the avatar by hacking in some CSS onto the gif and the canvas, but this resulted in glitchy animation.

Of course I could probably solve this by using a gif editor, but I thought I would mention it.

Edit : yes this did the trick : https://ezgif.com

It's all very nice work though :)

 


LLaMA2 Meta's chatbot released
by spydaz (AI News )
August 24, 2024, 02:58:36 pm
ollama and llama3
by spydaz (AI News )
August 24, 2024, 02:55:13 pm
AI controlled F-16, for real!
by frankinstien (AI News )
June 15, 2024, 05:40:28 am
Open AI GPT-4o - audio, vision, text combined reasoning
by MikeB (AI News )
May 14, 2024, 05:46:48 am
OpenAI Speech-to-Speech Reasoning Demo
by MikeB (AI News )
March 31, 2024, 01:00:53 pm
Say good-bye to GPUs...
by MikeB (AI News )
March 23, 2024, 09:23:52 am
Google Bard report
by ivan.moony (AI News )
February 14, 2024, 04:42:23 pm
Elon Musk's xAI Grok Chatbot
by MikeB (AI News )
December 11, 2023, 06:26:33 am

Users Online

363 Guests, 0 Users

Most Online Today: 447. Most Online Ever: 2369 (November 21, 2020, 04:08:13 pm)

Articles