html UI for chatbot

  • 16 Replies
  • 14648 Views
*

Merlin

  • Trusty Member
  • **
  • Bumblebee
  • *
  • 46
    • Skynet-AI
Re: html UI for chatbot
« Reply #15 on: December 08, 2011, 11:01:45 pm »
I would love to allow selectable TTS.

For number 3, there is a client side (Javascript) version of espeak;
https://github.com/kripken/speak.js

Demo:
http://syntensity.com/static/espeak.html

For tight integration, lip-synch should be driven by the audio. For loose integration, like what you might see in a cartoon, it could be driven by the returning text stream or be almost completely random as long as it runs while the audio is playing.

Playing the images is trivial. I am thinking about how to add more structure on top of that so that creation of the avatars can be standardized.



*

Bragi

  • Trusty Member
  • ********
  • Replicant
  • *
  • 564
    • Neural network design blog
Re: html UI for chatbot
« Reply #16 on: December 09, 2011, 08:51:16 am »
Ahh, great. I hadn't yet looked for a javascript version of espeak. Solves 1 more problem: server load.
From a first glance, this appears to be an automated port, so some testing will have to be done. By default, the espeak lib doesn't support lip-syncing (no viseme events), but there are phoneme events (in the form of callbacks), which can be converted into viseme events. Secondly,the espeak lib does this with callbacks, but that doesn't work in a web environment cause the audio is rendered first and then played, so either the callback data needs to be buffered (while it is being rendered), or the lib needs to be modified so that it includes an array with viseme events when audio playback starts.
Also, we'll need a reliable way to know when audio starts to play. I don't know how to do that in javascript, but I figured, if games can do it, it should be possible.

Quote
For tight integration, lip-synch should be driven by the audio. For loose integration, like what you might see in a cartoon, it could be driven by the returning text stream or be almost completely random as long as it runs while the audio is playing.

Playing the images is trivial. I am thinking about how to add more structure on top of that so that creation of the avatars can be standardized.
Well, I already have my system, which I will be porting to javascript so that both online and offline, the same avatars can be used. It's based on verbot's system (I added some things to make it more flexible).
It's structured, but still leaves a lot of freedom in the animations: there is no division like: eye-blink, nose move,....  but rather: idles, visemes, backgrounds,.. In combination with Z-order and transparency. eyes, hair, eye-brows,... those can be done by convention.

 


OpenAI Speech-to-Speech Reasoning Demo
by ivan.moony (AI News )
Today at 01:31:53 pm
Say good-bye to GPUs...
by MikeB (AI News )
March 23, 2024, 09:23:52 am
Google Bard report
by ivan.moony (AI News )
February 14, 2024, 04:42:23 pm
Elon Musk's xAI Grok Chatbot
by MikeB (AI News )
December 11, 2023, 06:26:33 am
Nvidia Hype
by 8pla.net (AI News )
December 06, 2023, 10:04:52 pm
How will the OpenAI CEO being Fired affect ChatGPT?
by 8pla.net (AI News )
December 06, 2023, 09:54:25 pm
Independent AI sovereignties
by WriterOfMinds (AI News )
November 08, 2023, 04:51:21 am
LLaMA2 Meta's chatbot released
by 8pla.net (AI News )
October 18, 2023, 11:41:21 pm

Users Online

304 Guests, 0 Users

Most Online Today: 346. Most Online Ever: 2369 (November 21, 2020, 04:08:13 pm)

Articles