[Freddy]

Time for an update, been a while. For some time now I have been developing my ElfScript language and this clip shows some examples of what it can do.

The idea is to teach Jess through chatting to her as well as making a lot of scripts. Here she learns variables and new patterns as we go along. So she remembers things you tell her like what a car or bicycle is. There's a couple of tricks like remembering what she was talking about when you say "It" - making teaching easier and more natural.

She can follow up on things that were mentioned earlier in the conversation and can tell you when she already knows something and then asks to learn something else about it.

So this chat is more about showing the features than being a sensible chat - I have a lot more scripting to do. Using the dictionary that I built (with thanks to Infurl for the links) she can make judgements on how to use a word or phrase. And she builds simple relations between words and phrases in her memory - like how she learns a couple of ways to get around in the following video.

In this example I also ask her who David Bowie was and she finds that out online. There were a couple of times she used the same acknowledgement phrase so I will give her a bigger pool of phrases to choose from for that part.

I suppose the other thing is that I have improved the lighting, she doesn't look so washed out as in some of my older videos. I've also learnt which Daz models work well - a lot of it regarding the textures for her skin etc.

In this video I've also introduced random facial movement, raising eyebrows, puckering etc. I had also written some code to do blinking and eye movement, but found one for sale that was better than mine for a good price. She can also track and watch the user as I tried to show at the start of the video.

My ElfScript also allows me to pass commands to the Unity player - so that I can trigger say, raised eyebrows, when she says something in particular. You can see it in the video at stages.

As this is in Unity, I have also been able to meet up with her in virtual reality, which is strange but cool ! This video is just about typed input, but I had some limited success with voice recognition, so you can actually talk to her and she will respond. I've used Google Voice Recognition API for that - but either it's got some work to be done, or my accent is confusing it, as it's all a bit hit and miss. I'm sure they will improve that, it is only in Beta at the moment.

I recorded the video at 1440p, but my movie maker only goes as far as 720p or something. It's not too bad though. Well that's me done with my rambling...over to Jess. Oh and you will see she knows her name



Transcript :

Freddy: hello jenny

Jess: My name is Jess actually.

Freddy: hello Jess

Jess: Hi there !

Freddy: got a favourite song

Jess: Secrets I think.

Freddy: who sings that

Jess: The Pierces.

Freddy: i like it too

Jess: Cat is so amusing in the video.

Freddy: a car is a good way to get around

Jess: I always like to learn new things.

Freddy: a bicycle is a good way to get around

Jess: Okay thanks.

Freddy: it is good for keeping you fit

Jess: Okay thanks.

Freddy: it is pretty cheap to buy

Jess: Okay thanks.

Freddy: it is pretty cheap to buy

Jess: Okay it is, I already knew it's that. What colour is it ?

Freddy: red

Jess: I didn't know that was it's colour.

Freddy: what is a bicycle

Jess: A bicycle is good way to get around and pretty cheap to buy.

Freddy: name a good way to get around

Jess: Okay, I know car and bicycle.

Freddy: who was David Bowie

Jess: David Bowie was a British musician, actor, record producer and arranger.

Freddy: who do you read

Jess: Dan Brown is my favourite authour.

Freddy: bye

Jess: I have nothing to say to you.

Freddy: cya

Jess: Cheerio.

[8pla.net]

I have a few suggestions.

On a new experimental interface, I suggest:

1.  Moving the textbox up across a (daylight) sky,
2.  Increase the length of the textbox to handle a bigger font
3.  Add a male text to speech to narrate the textbox input.

Backup the current interface.  It is excellent, especially
for a stand alone application.   My suggestions are
video related modifications.  I think they may make it
look like a television commercial which would be fun.

[Freddy]

Thanks

When I started out early this morning I had the intention of doing a recording of me in VR talking to her through voice recognition, so I was hoping to get my voice in there to show you guys. But after trying various more complicated sentences it was clear it's just not going to work yet.

It could cope with short phrases like "hello there" and "what are you doing" etc. but if I said something like "a fish is an animal" I would get it interpreted as "a phish is an annum" or some kind of nonsense like that. And even with my fuzzy pattern matching that was creating problems ! So I reverted back to typing for now. Maybe I will have to get elocution lessons. 

1) The text might be better high up, it's just more difficult to read on my screen like that - where it is now is a comfortable eye-line.

2) Are you on a laptop ? The text box is really quite large on my screen. Try viewing the video full screen to get some idea.

3) Could be a plan for future videos yes.

The interface I kept quite basic, the environment is the default one in Unity. As you've seen maybe, I have had her in a forest in the past. I was playing with some indoor interiors, but for this demo I decided to keep it simple and focus on the new features/improvements of the character. I think I might try making some scenes for her to appear in some time.

The television commercial is a fun idea

[Data]

The AI learning is neat Freddy, she grasped it first time and it feels quite natural as well. You did a great job on the characters appearance and animations too.

What else could I say, it's good  

[Art]

Very good, Freddy.

Actually in Full Screen the Text box is split by the red 'progress bar' at the bottom whereas just viewing as a smaller window on the site, was not shown in than manner.
I prefer to read text along the bottom or under the character rather than above one. Perhaps after so many years of most bots having their output windows beneath them it could just be one of those adopted preferences. Making the Output box have a whitish background and a black text and Jess's window just above it (if you were able to size it), might be a worthwhile combination.

Of note, I did like her facial animations, the eyebrows raising, and movements, plus her other gestures were also nicely done.

I too was waiting to hear your voice, but one problem might be that Google's Speech was fitted for that other English across the pond...maybe Amerish or similar. Try it without your accent...or is that even possible? Hmm... 

[Freddy]

The AI learning is neat Freddy, she grasped it first time and it feels quite natural as well. You did a great job on the characters appearance and animations too.

What else could I say, it's good

Thanks my friend. I did a lot of work on the learning side and there's different approaches that you can use depending on the situation you might find yourself in. I'm glad it seemed fairly natural. Jess' responses look better than mine as I made some sentence tidying routines for her. Being human I don't benefit from that   

I'm really pleased you like her appearance. A few things just clicked into place over the past couple of days, one of them was my tinkering with the environment light and the other was getting the eyes to look better - that was just a fluke as I flicked through various render settings.

[Freddy]

Actually in Full Screen the Text box is split by the red 'progress bar' at the bottom whereas just viewing as a smaller window on the site, was not shown in than manner.
I prefer to read text along the bottom or under the character rather than above one. Perhaps after so many years of most bots having their output windows beneath them it could just be one of those adopted preferences. Making the Output box have a whitish background and a black text and Jess's window just above it (if you were able to size it), might be a worthwhile combination.

Thanks Art. I see what you mean about the split - I'll nudge it up in future. I'll try black on white too. You can resize the player when you run it, but it's not laid out like a regular Windows form type affair.

Of note, I did like her facial animations, the eyebrows raising, and movements, plus her other gestures were also nicely done.

Makes her seem more alive doesn't it. There is a host of expressions to choose from as well, I just touched on a few.

I too was waiting to hear your voice, but one problem might be that Google's Speech was fitted for that other English across the pond...maybe Amerish or similar. Try it without your accent...or is that even possible? Hmm...

Well I set the locale to GB English, so I would have thought it would take care of it. It was really early in the morning and I wasn't at my peak. I'll play with it some more though. There is another option using Windows speech recognition - which I think is what powers Cortana - but unfortunately Unity broke that in some update, so people are waiting for them to fix it.

[8pla.net]

Yes, you are right, the full screen does show the font size larger.

In Unity, can you create a fraction of a second delay on the audio track? 
I may have noticed slight lip movement after the audio is finished.  I think
slightly sliding the audio track to the right of the video track on the timeline
may sync it up perfectly.

Sometimes, I add a fraction of a second to the beginning of the audio track.
So, I'll just offer this tiny tweak as feedback. My suggestions on this thread
are elective,  not constructive criticism in this case.  This is high quality work.

[Freddy]

Yes I noticed that too, at the end of some sentences. I think this is due to upgrading my graphics card as the decay on the lip movement morphs is tied to the fps.

It's not an audio track as such so I can't slide it. This version is using SAPI and it fires the phoneme events in realtime.

What I will do is try to limit video to 60 fps and that might fix it. Currently I think it is running fps quicker than needed and is messing with the phoneme decay. Or in my adjustments when moving from MaryTTS I made an error on one of the phonemes.

Or just move it to a fixed rated timer.

Well picked up