Ai Dreams Forum

Member's Experiments & Projects => General Project Discussion => Topic started by: Bragi on December 05, 2011, 02:28:19 pm

Title: html UI for chatbot
Post by: Bragi on December 05, 2011, 02:28:19 pm
So, I've got the backend for the chatbot server working and the front end is also working, but looking like crap. Here's the basic html that the asp.net server currently renders for a new chat session:


Code
<body>
    <div class="page">
        <header>
            <div id="title">
                <h1>Aici for</h1>
            </div>
        </header>
        <section id="main">       
			<div>
				<script src="/Scripts/Scroll.js" type="text/javascript"></script>
				<div style="overflow:scroll;" id="DivLogView"></div>
				<form action="/Html/Send" method="post">    
				    <p>
				        <input class="text-box single-line" id="Input" name="Input" type="text" value="" />
				        <input type="submit" value="Send" />
				    </p>
				</form>
			</div>
        </section>
    </div>
</body>
Very simple: the main  content has a div which contains the chatlog (currently server-side rendered, so it contains <p> elements for each log line, if anyone has a better idea, shout it out!) and a form with a single input box to send input statements.

-A first solution I can probably do is have a customisable css file so that the elements can be styled. (I'll have to add a class to the divLogView as well I suppose, anywhere else? )
-any other ways that this could be customized?
-Do other services (like pandorabots and the likes) allow you to completely customize the html?  Also, the html that I have, is this enough for someone to integrate it into their own site or is the chatlog history a bit of a broblem?

Thanks guys.
Title: Re: html UI for chatbot
Post by: infurl on December 05, 2011, 03:07:35 pm
Do you know how to write stylesheet transformations (XSLT)? It complements CSS in that it allows you to quickly and simply transform XML (or XHTML) by including an XSL file as well as or instead of your CSS file. Most browsers nowadays can run the XSL on the browser meaning that it only has to fetch the XML from the server for each page. It makes for a much faster website, as well as being easier to maintain.

For example, here's a page from one of my websites, which is sent from the server as XML:

http://understorey-network.org.au/family-index.xhtml (http://understorey-network.org.au/family-index.xhtml)

and the same page sent as HTML (the XSL stylesheet is run on the server instead of the browser):

http://understorey-network.org.au/family-index.html (http://understorey-network.org.au/family-index.html)

You shouldn't see any difference until you look at the page source code. It only needs one XSL file for the entire site.
Title: Re: html UI for chatbot
Post by: Merlin on December 06, 2011, 02:37:42 am
I use a set of divs for:
main (input),  avatar, current volley, log, alt HtmL/iframe (like if you do a web search) or content not in the log.
if you set these up via ids/CSS you can then either position them absolutely or show/hide them. If you are going to show these on someone else's web site you may want to wrap the whole thing in another div.
Title: Re: html UI for chatbot
Post by: Bragi on December 06, 2011, 08:06:14 am
@Infurl: I could perhaps use the xslt transform for the api inteface, which provides access to the chatbot for non html use. It accepts an input statement and returns an ssml file containing the output statement. The xslt could perhaps be used to filter out the text bit and display this in a browser instead of the full xml file.
Perhaps it could also be used for the log. For the html interface though, that'll be a bit tricky since it's initially rendered is in html anyway.

@Merlin: yes the idea would be to allow users to incorporate the bot on their own site, so it should be relatively easy to make it look the same as the host site. What you say about using a div for everything sort of makes sense. This is so that it can all be customized, including the position, correct? I hadn't thought about putting the input box and the whole thing inside another div.
-Also, what I have been thinking about: with the asp.net thingy, it's relatively easy to render a partial page, so excluding the <body> and stuff, bot only the relevant part. I was thiinking, perhaps you can use jquery (http://frinity.blogspot.com/2008/06/load-remote-content-into-div-element.html) to insert the chatbot content into a div on the user's site. If this would be a partial rendering (of the chatbot) and the divs are using 'standard' classes, wouldn't the chatbot automatically take over the style of the parent application? Or is this not proper and it's better to use iframes?
Title: Re: html UI for chatbot
Post by: Freddy on December 06, 2011, 03:12:06 pm
Quote
-Do other services (like pandorabots and the likes) allow you to completely customize the html?

SquareBear could probably answer this better than me.  But Pandorabots is pretty flexible, you can upload an html page of your own design.  I managed to get Haptek on a pandorabots page and also CrazyTalk.

And SquareBear's bot Mitsuku uses Flash...
Title: Re: html UI for chatbot
Post by: Bragi on December 06, 2011, 03:30:40 pm
So basically, they say: use these and these form posts in your html and then you can upload it, correct?
I could do that, I think.
The only thing I'm not certain about is this: for including the avatar: whenever the form-post returns, it also contains the audio and lip-sync info to play (if the correct post method is used). Is it possible for me to provide a reusable object that can be included in the custom html, or will that break the link with the post?
For info: the avatar is (or better said, will be)  implemented using a HTML 5 canvas element and lots java script to load the images from the server and play the audio sync data.
Title: Re: html UI for chatbot
Post by: Freddy on December 06, 2011, 04:18:15 pm
Quote
So basically, they say: use these and these form posts in your html and then you can upload it, correct?

Here's what they say : http://www.pandorabots.com/pandora/pics/pubhtml.html (http://www.pandorabots.com/pandora/pics/pubhtml.html)

Pretty simple but also quite flexible.  It's been a while since I did it, I hope SquareBear can shed more light.
Title: Re: html UI for chatbot
Post by: Bragi on December 06, 2011, 05:11:30 pm
Yep, I can do something similar. I'm currently using this razor thing (http://msdn.microsoft.com/en-us/gg618477) for rendering the html code. I could allow these to be modified.
Probably all for the future though, first getting things to work online.
Title: Re: html UI for chatbot
Post by: squarebear on December 06, 2011, 09:13:48 pm
Yes the HTML is totally customisable in Pandorabots. I have an example with a Haptek character, Flash application (the countdown) and graphics here:
http://bandore.pandorabots.com/pandora/talk?botid=8bd8ae6b3e345afc&skin=demo (http://bandore.pandorabots.com/pandora/talk?botid=8bd8ae6b3e345afc&skin=demo)
You just write your own HTML page and upload it.

However, I tend to use the API from Pandorabots in a Flash application I wrote for Mitsuku, as I do a few extra tricks within Flash itself like playing games, music, magic tricks and so on which I can't do in AIML.
Title: Re: html UI for chatbot
Post by: Merlin on December 07, 2011, 02:44:25 am
If you use divs and merge it with JavaScript or Jquery it should take on the default look of the page. Iframes also work. there are pros and cons of each approach.

For your avatar/audio, could you target the return to a different iframe? In some set-ups, I have made each component in a different iframe. In others, I put everything in sets of Divs that  I write dynamic html to.
Title: Re: html UI for chatbot
Post by: Bragi on December 07, 2011, 07:55:22 am
Ok, so I had a night's sleep on this and things are getting a bit clearer. First, I can actually use the same trick as pandora for customizing the html (simple string replace). All I need to do is replace the !CHATLOG! (or what was it), and add an extra !AVATAR!, for, yep you guessed it. This way, the HTML customization can be compatible with pandora bots.
I can also allow the css to be customizable in case that the HTML can remain the same (for Iframes) and there's a separate call for doing partiall rendering so that the jquery trick also works.  Finally, there's the api, which should allow for something similar like the flash app. I think that should cover all the bases.

Quote
For your avatar/audio, could you target the return to a different iframe?
You mean, with a single form-post also update another IFrame. Is that possible? Don't know how to do that.
Title: Re: html UI for chatbot
Post by: Merlin on December 07, 2011, 02:47:09 pm
Don't know if it would be helpful for you, but the HTML form tag supports TARGET and onSubmit. If you are going to to render the avatar on your own, you might want to use these. For me, I have used it to prevent the form from flashing and I have separated the functions of storing the log, calling the AI and updating the Avatar.
Title: Re: html UI for chatbot
Post by: Bragi on December 07, 2011, 03:23:22 pm
Ah, cool. didn't know that.
In case you might not yet have noticed: html == my Achilles.
Title: Re: html UI for chatbot
Post by: Merlin on December 08, 2011, 04:16:48 pm
Quote from: Bragi on December 06, 2011, 03:30:40 pm
For info: the avatar is (or better said, will be)  implemented using a HTML 5 canvas element and lots java script to load the images from the server and play the audio sync data.

I have been thinking of doing a javascript/canvas/avatar renderer also.
Something to replace MS Agent and Haptek, but totally based on HTML5 web standards.
Maybe we should put our heads together on this.

Title: Re: html UI for chatbot
Post by: Bragi on December 08, 2011, 06:08:43 pm
Good idea.
I figured that the actual avatar rendering isn't that hard: just load all the images in a couple of arrays, fire off a timer (javascript games use a different technique I believe, I have it bookmarked somewhere) and that's basically it.
The tricky bit is in the audio rendering and lip sync. I figured there were 3 options:
1: use a standard library already available on the client. which means the managed sapi library. (lip sync is done by events)
2: use a similar technique as google: have audio rendered on a server and send them to the client together with the lip-sync info (which is played on the client through javascript and the timers).
3: write a javascript based, synth speech generator that can run locally. (lip-sync is done with same technique as server based audio).
Now, option 1 is a bit limiting, but more important: the managed SAPI version is buggy and it appears to be a dead project (bug reports from a couple of years ago are still very much present and they are no longer providing any technical support for the lib). I don't know what they are up to...
3: should be possible, but I'm no java expert.
so, I am going with option 2: have the audio rendered on the server, send it to the client and play the mp3. I was planning on using an espeak derivative for this.
Title: Re: html UI for chatbot
Post by: Merlin on December 08, 2011, 11:01:45 pm
I would love to allow selectable TTS.

For number 3, there is a client side (Javascript) version of espeak;
https://github.com/kripken/speak.js (https://github.com/kripken/speak.js)

Demo:
http://syntensity.com/static/espeak.html (http://syntensity.com/static/espeak.html)

For tight integration, lip-synch should be driven by the audio. For loose integration, like what you might see in a cartoon, it could be driven by the returning text stream or be almost completely random as long as it runs while the audio is playing.

Playing the images is trivial. I am thinking about how to add more structure on top of that so that creation of the avatars can be standardized.


Title: Re: html UI for chatbot
Post by: Bragi on December 09, 2011, 08:51:16 am
Ahh, great. I hadn't yet looked for a javascript version of espeak. Solves 1 more problem: server load.
From a first glance, this appears to be an automated port, so some testing will have to be done. By default, the espeak lib doesn't support lip-syncing (no viseme events), but there are phoneme events (in the form of callbacks), which can be converted into viseme events. Secondly,the espeak lib does this with callbacks, but that doesn't work in a web environment cause the audio is rendered first and then played, so either the callback data needs to be buffered (while it is being rendered), or the lib needs to be modified so that it includes an array with viseme events when audio playback starts.
Also, we'll need a reliable way to know when audio starts to play. I don't know how to do that in javascript, but I figured, if games can do it, it should be possible.

Quote
For tight integration, lip-synch should be driven by the audio. For loose integration, like what you might see in a cartoon, it could be driven by the returning text stream or be almost completely random as long as it runs while the audio is playing.

Playing the images is trivial. I am thinking about how to add more structure on top of that so that creation of the avatars can be standardized.
Well, I already have my system, which I will be porting to javascript so that both online and offline, the same avatars can be used. It's based on verbot's system (I added some things to make it more flexible).
It's structured, but still leaves a lot of freedom in the animations: there is no division like: eye-blink, nose move,....  but rather: idles, visemes, backgrounds,.. In combination with Z-order and transparency. eyes, hair, eye-brows,... those can be done by convention.