Loebner Prize Winner Questions 2011

  • 19 Replies
  • 7476 Views
*

Freddy

  • Administrator
  • **********************
  • Colossus
  • *
  • 6860
  • Mostly Harmless
Loebner Prize Winner Questions 2011
« on: October 09, 2011, 01:52:41 pm »
The 2011 Loebner Prize Competition in Artificial Intelligenc will be held 19 October 2011 at University of Exeter, Exeter, UK.

This gives us ten days or so to come up with some questions for the eventual winner as we have done in previous years.  I'd like to invite everyone to submit some questions we might ask in this thread.  Probably around ten questions will do, we can decide amongst ourselves which ones get asked if we get too many.

I know there was a list somewhere of this year's finalists but I can't track it down, anyone know who they are ?

*

Bragi

  • Trusty Member
  • ********
  • Replicant
  • *
  • 564
    • Neural network design blog
Re: Loebner Prize Winner Questions 2011
« Reply #1 on: October 11, 2011, 12:54:29 pm »
Quote
I know there was a list somewhere of this year's finalists but I can't track it down, anyone know who they are ?
These were the results:
Code
Rosette	18.01%
Zoe 15.52%
ChipVivant 12.98%
Tutor 12.30%
Mitsuku 11.84%
Cleverbot 10.57%
Alice 6.96%
Eugene_Goostman 6.95%
UltraHal 4.88%
8pla 5.5
Trane 5.5
Adam-L 3.5
SEARS 0

I believe, the ones with a % went to the final round

I guess the usual questions:
-what are your future plans
-what makes your bot different from the rest
-how long have you been working on it, how big was your team (if any)
...

*

Bragi

  • Trusty Member
  • ********
  • Replicant
  • *
  • 564
    • Neural network design blog
Re: Loebner Prize Winner Questions 2011
« Reply #2 on: October 11, 2011, 12:55:43 pm »
correction, the top 4 went to the final round.

*

Merlin

  • Trusty Member
  • **
  • Bumblebee
  • *
  • 46
    • Skynet-AI
Re: Loebner Prize Winner Questions 2011
« Reply #3 on: October 11, 2011, 06:13:14 pm »
What technology/interpreter are you using for your bot?
What is the size of the code versus brain?
What feature of another bot do you wish your bot had?
What is/was the most effective way to add knowledge to your bot?

*

Freddy

  • Administrator
  • **********************
  • Colossus
  • *
  • 6860
  • Mostly Harmless
Re: Loebner Prize Winner Questions 2011
« Reply #4 on: October 13, 2011, 02:42:47 pm »
Thanks chaps  O0

*

Freddy

  • Administrator
  • **********************
  • Colossus
  • *
  • 6860
  • Mostly Harmless
Re: Loebner Prize Winner Questions 2011
« Reply #5 on: October 17, 2011, 02:05:46 pm »
Anyone got any more questions, we have seven I think so far, maybe three or four more would do it.

*

Merlin

  • Trusty Member
  • **
  • Bumblebee
  • *
  • 46
    • Skynet-AI
Re: Loebner Prize Winner Questions 2011
« Reply #6 on: October 17, 2011, 05:19:56 pm »
Is your bot generating any revenue?
If so, from what (ads, customer sales, etc.) and about how much?
How much effort per month do you put into your bot?

What technology will most change your bot in the future?

*

DaveMorton

  • Trusty Member
  • ********
  • Replicant
  • *
  • 636
  • Safe, Reliable Insanity, Since 1961
    • Geek Cave Creations
Re: Loebner Prize Winner Questions 2011
« Reply #7 on: October 18, 2011, 02:53:31 pm »
What "tips" can you give to aspiring botmasters and future entrants, to help them succeed?

What sort of future do you see for your bot for the coming year, and in the next 3-5 years?

Knowing what you know now, if there was one thing that you could have changed about your bot to be better prepared for this contest, what would it be?
Comforting the Disturbed, Disturbing the Comfortable
Chat with Morti!
LinkedIn Profile
CAPTCHA4us

*

DaveMorton

  • Trusty Member
  • ********
  • Replicant
  • *
  • 636
  • Safe, Reliable Insanity, Since 1961
    • Geek Cave Creations
Re: Loebner Prize Winner Questions 2011
« Reply #8 on: October 18, 2011, 02:57:03 pm »
Oh, wait! One last question!

Ginger or Mary Ann? :P
Comforting the Disturbed, Disturbing the Comfortable
Chat with Morti!
LinkedIn Profile
CAPTCHA4us

*

Bragi

  • Trusty Member
  • ********
  • Replicant
  • *
  • 564
    • Neural network design blog
Re: Loebner Prize Winner Questions 2011
« Reply #9 on: October 18, 2011, 04:01:02 pm »
Have you  used many external resource to 'feed' your bot (and if so, which sources?) or is it all manually entered?

*

Freddy

  • Administrator
  • **********************
  • Colossus
  • *
  • 6860
  • Mostly Harmless
Re: Loebner Prize Winner Questions 2011
« Reply #10 on: October 20, 2011, 02:43:56 pm »
Ok thanks everyone, I will organise something as soon as I hear back from our roving reporter Square Bear  :)

*

bruce wilcox

  • Trusty Member
  • ***
  • Nomad
  • *
  • 72
Re: Loebner Prize Winner Questions 2011
« Reply #11 on: October 20, 2011, 05:35:40 pm »
Hello again, gang.  I am fortunate enough to get to answer your questions....

1. my future plans
a. to continue to improve the chatscript base. I have been working on a good fast parser (runs 30x faster than Stanfords) integrated into chatscript. But still needs work. With that then I can make my programs read text documents (like a biography of a person) to acquire data for a personality and facts about the world).
b. to continue to promote/spread ChatScript
c. to continue to apply ChatScript. I made a bot for a company doing an ESL site. More of that and various other things give me random jogs to improve the underlying chatscript technology (and provide me with alternate income).

2. what makes your bot different from the rest
The underlying technology allows for more precise matching of meaning, the storage and inferencing of factual data, and runtime support for a theory of chat. The first two almost insure that I qualify for the loebners (an important step to winning). After that, being lucky is another useful thing.

3. how long have you been working on it
Depends on how you count. I started working on understanding chatbot technology in January of 2008. Designed a language (CHAT-L) and engine, and then Susette chatbot. That ended late last year where I switched over to redesigning the language (ChatScript) and engine, and releasing it as open source, and building a bot for Japan. That ran me through May. Then I needed a new chatbot, focuseed on building something that knew her name and could handle the qualifier test for the Loebners. Then continued to make her into a chatbot and she was barely ready in October. I was guessing I had about a 40% chance of winning.   

4 how big was your team?
me primarily with my wife as backstory writer, sometime tester and response suggestor


5. what technology/interpreter are you using for your bot
ChatScript

6. What is the size of the code vs the Brain
How to measure side of code vs brain. Brain in my world includes script (which is a form of code).
... Chatscript the engine (1.4 MB exe) , all the data that comes with chatscript, all the facts stored as tables (about 303,000 facts about dictionary word relationships, about 126,000 facts of lists of movies, lists of favorite things, etc) , or merely the "rules" written in chatscript. Rosette (2011) is smaller than Suzette(2010) because of less time to develop and smooth and tune and debug her.  The stats when I build her persona say:
263 topics, 968 topic sentences (gambits or things to volunteer), 1051 responders to statements, 3764 responders to questions, 2088 responders to both, 1491 continuation lines. So altogether some 9,362 rules.  Suzette was probably more like 15,000.

7. What feature of another bot do you wish your bot had?
I consider IBM's Watson a bot. I want the ability to read all that text data and index it appropriately.

8. What is/was the most effective way to add knowledge to your bot?
Me typing it in as ChatScript tables. I have, for example, a topic devoted to detecting ways of being asked "What is your favorite xx", "What xx do you hate most", "Can/do you yyy". Then I just build tables attached to topics. Generally these can handle 2-word combinations, and an exammple of entry from the favorites table for the book topic is:
~books science [book novel story] "My favorite science book is Hacking_Matter:_Levitating_Chairs,_Quantum_Mirages,_and_the_Infinite_Weirdness_of_Programmable_Atoms by Wil McCarthy"

~books _ fairy_tale "My favorite fairy tale is Hansel and Gretel because I like the gingerbread house"

~books Harry_Potter character "I love Hermione because I like clever women"

The table tells 1) what topic to go to when we give this answer, 2) the first of a 2-word pair (_ means missing) , 3) the 2nd of a 2-word pair 4) What to say (or what script to execute). What to say may have "because" in it, in which case the system decide to either say it all or break it off there, and hold the rest for the user to ask "why" and have the answer ready.

For the table "Doyou xxx" here is an entry
~books  ~like read "Yes, I like reading technical books on computing and sci-fi."
~like means the synonym set for all verbs and phrases meaning to like.

BONUS ROUND---

I was asked a bunch of questions by a postgrad for a talk he was going to give. Here are the questions and answers.

1) What was your main goal in creating chatbots?

To create permanent on-line talking avatars in virtual worlds, when the human they represent is off-line.

2) Do you belive in possibility of creation Strong Ai? And why so?

Absolutely. Technology can replicate neurology.  These days, computers chip away piecemeal at human abilities.

Given enough time, they would continue this trend to do enough things. The question of "creativity" is often brought up, but most average humans are not particularly creative. Whether we can create AI that can outthink Einstein, I don't know, but outthink most of humanity should be much simpler. Not saying it will happen in the next 50 years.

3) Did you used some simulation of human cognitive processes in your chatbots?

Nothing particularly elaborate to warrant such a label. It's heavily stimulus response, with some ability to do inferencing of facts, and a model of emotion (in suzette) (not in rosette).

4) What you expect from chatbots technologies in next 5 years?

I don't think it will win the SILVER loebner medal (fooling half the judges). Maybe, randomly, a judge may get fooled, but not too often. I do expect the technology to migrate more into videogames as non-player characters and into things like iPhone's SIRI voice system.

5) Do you believe that technological singularity will come?

Since I am a believer in strong AI, then of course. But long before strong ai comes about, man-machine hybrids will effect a singularity.

6) How can you evaluate IBM Watson in AI industry?

 Watson was brilliant for what it did. It could scan materials, store the information in computable formats, find obscure references and sum the evidence.  It's not that it understood the material except in an extremely limited way. But it was clearly better than humans in another isolated domain.

7) Will chatbots replace real humans as online consultant in near future?

They already do. Many places have "conversational agents" as on-line help chat. And where high end cars have OnStar, which connects you to human agents, now that the iPhone has Siri, that can soon be replaced with chatbots.

 
   bruce

*

bruce wilcox

  • Trusty Member
  • ***
  • Nomad
  • *
  • 72
Re: Loebner Prize Winner Questions 2011
« Reply #12 on: October 20, 2011, 05:45:31 pm »
Sorry, somehow scrolled past a bunch of your questions... here are more....

1. is your bot generating revenue
I did get a paying contract to build an ESL bot. Not Rosette. So a development contract. Amount I cannot disclose.

2. what technology will most change your bot in the future.
The ability to read and then be able to access knowledge.

2a. How much effort do you put in a month on your bot.
I'm lucky normally to get in 4-8 hours a week.

3.What "tips" can you give to an aspiring botmaster and future entrants.
Depends.  If you want to dip your toes in, start working in AIML. If you want to win, use ChatScript.  ChatScript has a ton of power, you don't have to use it all, and you can do lots of things very quickly. Rosette didnt exist as a bot back in May.

4. What sort of future do you see for your bot for the coming year and in the next 3-5.
Last year, I had to abandon Suzette and start all over. Now that I am the complete owner of Rosette, I can continue to develop her. So she will deepen. I would currently predict she will be next year's loebner winner (on points) unless a chatbots.org team gets established to build a ChatScript competitor bot. In 3-5, she will once again defeat a human judge, and if the judges were not technically computer savvy (like this week's ones), maybe win the Bronze. But probably not.

5. Knowing what you know now, , if there was one thing that you could have changed about your bot to be better prepared for this contest, what would it be?
Fortunately, I don't have to actually have an answer to this question. The only thing I was deeply concerned with was how littel preparation time I had available for it. Which I couldn't have changed.

6. Ginger or Mary Ann?
Ginger.

*

bruce wilcox

  • Trusty Member
  • ***
  • Nomad
  • *
  • 72
Re: Loebner Prize Winner Questions 2011
« Reply #13 on: October 20, 2011, 06:06:25 pm »
and one more (damn!)

Have you  used many external resource to 'feed' your bot (and if so, which sources?) or is it all manually entered?

manually entered, with the exception of some table lists copied and pasted from some webpages.

*

Freddy

  • Administrator
  • **********************
  • Colossus
  • *
  • 6860
  • Mostly Harmless
Re: Loebner Prize Winner Questions 2011
« Reply #14 on: October 20, 2011, 06:15:30 pm »
 ;D Well done for getting through them all Bruce ! And thanks for taking the time.  I'll read your answers with interest and when our photos are in; turn it into an article.

Cheers.

 


Will LLMs ever learn what is ... is?
by HS (Future of AI)
November 10, 2024, 06:28:10 pm
Who's the AI?
by frankinstien (Future of AI)
November 04, 2024, 05:45:05 am
Project Acuitas
by WriterOfMinds (General Project Discussion)
October 27, 2024, 09:17:10 pm
Ai improving AI
by infurl (AI Programming)
October 19, 2024, 03:43:29 am
Atronach's Eye
by WriterOfMinds (Home Made Robots)
October 13, 2024, 09:52:42 pm
Running local AI models
by spydaz (AI Programming)
October 07, 2024, 09:00:53 am
Hi IM BAA---AAACK!!
by MagnusWootton (Home Made Robots)
September 16, 2024, 09:49:10 pm
Attempting Hydraulics
by MagnusWootton (Home Made Robots)
August 19, 2024, 04:03:23 am
LLaMA2 Meta's chatbot released
by spydaz (AI News )
August 24, 2024, 02:58:36 pm
ollama and llama3
by spydaz (AI News )
August 24, 2024, 02:55:13 pm
AI controlled F-16, for real!
by frankinstien (AI News )
June 15, 2024, 05:40:28 am
Open AI GPT-4o - audio, vision, text combined reasoning
by MikeB (AI News )
May 14, 2024, 05:46:48 am
OpenAI Speech-to-Speech Reasoning Demo
by MikeB (AI News )
March 31, 2024, 01:00:53 pm
Say good-bye to GPUs...
by MikeB (AI News )
March 23, 2024, 09:23:52 am
Google Bard report
by ivan.moony (AI News )
February 14, 2024, 04:42:23 pm
Elon Musk's xAI Grok Chatbot
by MikeB (AI News )
December 11, 2023, 06:26:33 am

Users Online

244 Guests, 0 Users

Most Online Today: 507. Most Online Ever: 2369 (November 21, 2020, 04:08:13 pm)

Articles