[Tony Ellis]

Hi Ai friends

I been away for a few years and in this time I have built a fully operation full scale robot (just over 4 foot tall) the ALTAIR EZ:1 development robot. Its taken years to develop all the mechanical/electronics and firmware, but it is worth it as the robot is now pretty impressive.

I am now looking to collaborate with people to assist in adding some (rudimentary) Ai and chatbot type features and I remembered how friendly this forum is and also the great expertise available here. I am wondering if anyone wants to or knows of any talented people (with the required skills) that would like to join in on such an exciting venture?

Here is the EZ:1 (development robot) and the EZ:2 (commercial version)



Here is a pic of EZ:1 head



The top camera/sensor does the following:

Face recognition (up to 100 different faces) – high accuracy and works in most lighting conditions.
Face recognition on TV/media - the system can not only recognize its owner and all their friends and family but also TV programs and TV celebrities from the screen in real time.
Face detection – can track (position of) faces in real time.
Face direction estimation - estimates the direction of detected faces.
Estimation of facial expressions - Five different emotion expressions:
1) Happiness
2) Surprise
3) Anger
4) Sadness
5) Neutral
Human body detection - detect human bodies in view.
Gender estimation – pretty accurately estimates a face (users) gender.
Age estimation – pretty accurately estimates a face (users) age.
Gaze estimation - can track what direction the persons eyes are actually looking at.
Hand detection – using this for hand gesture applications.

The lower camera does:

Object and colour recognition

Various other sensors are indicated on the photo.

I developed SMART servos with have amazing torque (worm driven so no holding current) and virtually silent in operation - these can be seen in the 5 DOF arms/claws.

The main locomotion system is PID based, so it moves very accurately and can learn floor/area mapping.

I would welcome any input from you guys on this - I have got the really cool robot, but now need to add the Ai functionality which would make the robot awesome!

Thanks

Tony

[LOCKSUIT]

I have the full AI algorithm, are you willing to create it? It's simple really. I'm also about 2 days away from finishing my Master Diagram, which also abruptly shows my whole blueprint.

[Art]

Welcome back Tony! Yes, your presence has been missed but it's nice to see that the wheels have kept on turning with development. It looks much more advanced than the previous one as I recall.
Hopefully you'll connect with at least one of more individuals who have documented expertise in the fields that you / your bot require.

There are many robotics groups & forums online that could possibly assist if some here can't provide either the type of assistance or enough assistance. Just google robotics groups.

Best of luck and keep us posted on your progress.
- Art -

[Tony Ellis]

Thanks Art, its good to be back on this great forum - much to catch up on. The robotics forums usually are all about the build (mechanics, electronics etc) and rarely is their Ai expertise as there is here!  I have a brilliant functioning robot, but now I need to add the Ai bit to make it truly useful/functional.

I am looking to create a kind of Ai module (software) that you can keep adding functions to and the input/output is the robot itself. If I can find the right people to achieve this, then I would give them access to my robot tech so they could produce their own robot. Here is the EZ:2 and Dewey (from the film Silent Running) and an early EZ:1 doing object recognition.

[Freddy]

Welcome back Tony, amazing work. I hope you can find someone to work with

Does either have a voice ?

[ivan.moony]

Hello Tony

What activities would you like your robot to exhibit? I see a tablet attached to it.

[Tony Ellis]

The robots talk (using TTS) and use Dragon for speech recognition. Here is a video of an early ALTAIR robot doing object recognition



These were the old arm design and were a bit springy, the new arms are much more stable now thanks to the linear actuators in the elbows.

The EZ:1 development robot is Wifi linked to a host PC while the EZ:2 has the option of a body mounted tablet.

The sort of features that I would like to see

1) Conversation engine (I know this is not easy)

2) Use natural speech to ask the robot general questions, set reminders and timers, get news, sports and weather information etc and more

3) Learning - As its user requests actions to occur with the robot, it learns habits and then starts using these habits to then do things for its user automatically

4) Emotion engine - reactions/interactions from the data processed by the on-board emotion sensing sub-system

5) Human/robot interaction engine

Tony

[ivan.moony]

I don't know how much you are into programming, but these are some of my insights, in case you're interested:

The sort of features that I would like to see

1) Conversation engine (I know this is not easy)

Might be easier than you think. Other users could give you more informations on this.

2) Use natural speech to ask the robot general questions, set reminders and timers, get news, sports and weather information etc and more

A kind of tricky demand, depends on the amount of the robot independence.

The easiest thing to do is to pass an audio from microphone to Google Now engine. Google Now is built in their search pages and is available for everyone. Voice input is ok, it can show relevant web page search results depending on a common sense questions, which can be shown on the tablet. But extracting individual informations could be tricky. Maybe you could restrict results just to a few quality pages of your choice like weather cast, sports or world news to avoid that step of showing a lot of search results and surf directly to relevant content to show it on the tablet.

The more complicated task would be parsing natural language through Stanford parser (http://nlp.stanford.edu:8080/parser/). Just make sure if it can be run on your operating system and check the licence to see if it fits you. After parsing you can route demands to different algorithms (light switching, browsing web, ...), depending on input. But I predict a lot of work and special cases here if you don't own an AGI solution.

3) Learning - As its user requests actions to occur with the robot, it learns habits and then starts using these habits to then do things for its user automatically

A lot of adjustable defaults, if I may. Those actions after routing natural language requests can have a bunch of parameters, which you can pre-assume after initial user demands.

4) Emotion engine - reactions/interactions from the data processed by the on-board emotion sensing sub-system

I don't know if you can do it without AGI. And AGI is not here yet.

5) Human/robot interaction engine

See under 2)

You might want to check what's happening on "OpenCog" user group. They are a bunch of science doctors (and some floating satelites), building an AGI. They've done some work (including emotional stuff you require, ask for openPsi) and now they are after a human interaction system, if I understood correctly (they are testing it on a virtual 3D face made with Blender, ask for Eva). They work at a slow pace, but you can post a question there, maybe they would be interested in helping you (or sucking you into their own system). They work on Linux, I don't know if that satisfies you. You can ask a few questions there, it can't hurt (https://groups.google.com/forum/#!forum/opencog).

[Tony Ellis]

Thanks @ivan for your insights

I don't know if you can do it without AGI. And AGI is not here yet.

I have all the emotion sensing already working great here is the prototype before being added to the EZ:1



The sub-system send faces recognised, emotion, gender, age, head/face x/y coordinates and distance to the master robot controller in the ALTAIR robot. At present the robot shows emotion (back) via the eyes, mouth and arm/claw movements. Here are some of the eye displays for various robot emotions.



I looking to make the emotion engine more sophisticated with Ai functionality.

Tony
 

[Tony Ellis]

After the problems of some folks not seeing my photos (from google images), I have replaced all the pictures on this thread (now from Photobucket) so hopefully they are visible to the whole forum.

Here is the EZ:1 having its head put back on after the emotional sensor (enclosure with the vent holes in the upper head) was upgraded. There is a lot of wiring in an advanced robot!



Tony

[ivan.moony]

So detecting emotions is done? Nice, I like it.

But how to respond to them? Are you planning some chart or something? A response could be a thousand things, even if you know what emotion to output. But maybe I'm thinking too big again... It's a professional deformation of mine

For a sake of simplicity, a response for negative emotions could be simply:
* Please don't be (angry | sad | melancholic | ...)

and for positive ones:
* I like when you are (happy | groovy | singing | ...)

But if you want your bot to chat it out in a smarter response, I don't know the current stage of technology evolution. You know, chatbots are pretty simple to program, you just use an interpreter for AIML (Artificial Intelligence Markup Language). You can build custom AIML file, expressing your bot's personality (I think it takes a few thousands of responses in AIML for quality chatting), or you can use ready made ones, freely available on the web. I think that the current top technology name is AIML 2, but I'm not sure if the standard has implementation of emotions.

[Tony Ellis]

As some of you know, I have been working on linking a Pandorabot with (unlimited) speech input and output, and have found a way to use the speech recognition in cortana.

With cortana SR input I can say anything I want to the chatbot as this video shows - its all a bit rough as it is one of my first (vocal) chats with DJ's Pandorabot and I could not think of what to say!

Here is the video:



Tony

[Korrelan]

Very cool bot extension.

Going to interesting to see it coming from the actual bot with a few related arm gestures etc.

I'm sure many chatbot programmers/ users will be interested; especially if you can find a way to easily integrate the emotion, vision, motion, etc.

 

[Art]

Tony,
With all due respect, her speech responses seemed like she was a bit addled or slow. Hopefully her speech can be sped up or tweaked in the settings or some such.

Otherwise...great start.

Conversational bot flow and topic flow are the thorns of botmasters everywhere. It takes time to rummage through many, many lines of scripted responses and editing them. It's a labor of love for some and not so much for others, I imagine.

 

[Tony Ellis]

Art, that voice is dreadful! It was the very first attempt after putting it all together and the demo used one of the default TTS voices that unfortunately runs slow for some reason? I will be adding some Cereproc voices to that Win10 computer which will improve it greatly.

Tony