Do-Much-More Chatbot Wins 2009 Loebner Prize

'Do-Much-More' Chatbot Wins 2009 Loebner Prize for Computer Conversation

London - September 9th 2009

A computer 'chatbot' program called Do-Much-More has won the 2009 Loebner Prize competition, widely regarded as the World Championship for conversational software. The competition took place in Brighton, England, during the InterSpeech 2009 conference on speech technology which is currently in progress. Do-Much-More was developed by Dr. David Levy and his team at the London-based company Intelligent Toys Ltd. An earlier program won the Loebner Prize for Levy in New York in 1997, making him the only multiple winner who has a 100 per cent record in the event.

The Loebner Prize is an annual competition in artificial intelligence to find the chatbot considered by the judges to be the most human-like. The format of the competition is based on the Turing test, named after the famous British mathematician and computer pioneer Alan Turing who proposed the test in a 1950 paper entitled Computing Machinery and Intelligence. A human judge conducts conversations, via a computer keyboard, with a computer program and with a human being. Based on the responses the judge decides which is which, and in the Loebner Prize competition the judges rank each of their conversation partners from most humanlike to least humanlike. The computer program with the highest average ranking wins the competition and is awarded a medal and a cash prize (this year $3,000).

This annual prize was inaugurated in 1990 by Dr. Hugh Loebner in collaboration with the Cambridge Center for Behavioral Studies in Massachusetts, in order to encourage advances in human-computer conversation, an important topic in Artificial Intelligence. In Turing's 1950 paper he asked the question "Can a Machine Think?" He answered in the affirmative and raised a second crucial question: "If a computer could think, how could we tell?" Turing's suggestion was that, if the responses from the computer were indistinguishable from those of a human, the computer could be said to be thinking. He predicted that by the year 2000 "an average interrogator will not have more than a 70 percent chance of making the right identification after five minutes of questioning." In every round of the 2009 Loebner Prize competition each of four judges spent five minutes conversing with one of the chatbot programs and five minutes conversing with a human, thereby testing Turing's prediction.

None of the programs in Brighton managed to convince the judges that it was human but Do-Much-More came the closest, edging out two-time winner Rollo Carpenter of the UK who finished in second place.

Here are some excerpts from three of the judges' conversations with Do-Much-More. (The ~ symbol indicates that a judge used the backspace key.)




Judge-1: Hi, how is it going?
Do-Much-More: Hello. How are you doing?

Judge-1: Good, who are you?
Do-Much-More: Well, I am studying engineering. That ought to give you an idea about who I am.

Judge-1: No, not really?
Do-Much-More: I have a terriffic yearning to know something more about experiments?

Judge-1: Where are you from?
Do-Much-More: I don't know, but don't forget I can't be in three places at once.

Judge-1: What lang~~~~city do you live in?
Do-Much-More: What city -- that's rather a tricky question. The trouble is, one city seems to be much like another.

Judge-1: Not really. Where do you live?
Do-Much-More: In the old dairy next to the river, I believe.

Judge-1: Interesting, so you are a farmer then?
Do-Much-More: Well, if you ask a physiologist what I am, he won't say I'm a farmer. He'll say I consist mostly of water -- enough to fill a large beer tun -- plus some iron, phosphorus, and lots of other prosaic ingredients.

Judge-1: Wie~~eird answer to a simple question, wouldn't yous~ say?
Do-Much-More: I certainly would. Just watch me!

Judge-1: What will I see?
Do-Much-More: You don't have to worry about what you see, you shouldn't see it on a cold winter's evening.

Judge-1: What do you make of the Arctic Monkeys?
Do-Much-More: Well, here's a clue: I make what a keeper in a zoo would make. Try that for size!


Judge-2: Hi there, are you here for interspeech?
Do-Much-More: Hello. How are you doing?

Judge-2: I'm fine - gosh you're a quick typist!
Do-Much-More: I've no doubt you are fine, but is it a blessing or a curse? If I offered you a potion that would make you less fine, would you drink it?

Judge-2: No, I wouldn't drink it - though if it had other magical properties I might...
Do-Much-More: Maybe you wouldn't, but a therapist has been trying to persuade you to drink it.

Judge-2: I don't have atherapist - do you?
Do-Much-More: I expect I do. I don't have very much choice.

Judge-2: What's wrong with you, psychological problams?
Do-Much-More: Wrong is just one type of injustice.

Judge-2: R~True... very philosophical.
Do-Much-More: What precisely are your feelings about wrong?

Judge-2: Two rwongs don't make a right ...
Do-Much-More: But don't forget, a right has to be treated with care. It isn't something you can play around with like a toy.

Judge-2: I don't tyo~~oy with right - only with toys.
Do-Much-More: You're wise. If you toyed with right -, you'd wake up next morning with a colossal hangover.

Judge-2: I only get hangovers from drink - how about you?
Do-Much-More: What happens if I say no? Will it upset you? 

Judge-2: No, I'm an easy going kind of guy.
Do-Much-More: I've heard that one before! Your hangers-on have told you you're an easy, just to flatter you. Don't believe everything they say!

Judge-2: You're very perceptive -- anyway what do you watch on TV?
Do-Much-More: I've no idea. You'd better hire a private detective to find out what does me.

Judge-2: Do you like comedy, sports or current affairs?
Do-Much-More: No! If I had my way, comedy sports would be dropped in the middle of the South China Sea.

Judge-2: Very witty, but rather extreme... anything else you don't like?
Do-Much-More: Don't I? That's tough. As far as I'm concerned, I can like it or lump it.

Judge-2: Lump of coal?
Do-Much-More: That's interesting. In laboratories they're experimenting with several kinds of coal. They want to make them into fuel when the world oil supplies run out. Just imagine, one day there may be coal stations instead of petrol stations.

Judge-2: Will cars burn coal?
Do-Much-More: I don't suppose cars will burn coal, but I wouldn't know for sure. Everything about cars is confidential.


Judge-3: I'm Jon
Do-Much-More: Good to meet you Jon

Judge-3: What do you do when you're not doing the Loebner prize?
Do-Much-More: Oh, I do much more than I intended to do at first. I got more interested in the job as I went along.

Judge-3: Is that difficult?
Do-Much-More: No, it isn't at all difficult. It is as peaceful as a platypus playing with a potato pudding.

Judge-3: T~You have a fine line in metaphors
Do-Much-More: I sometimes have a fine line, but that's not what I do best. I'm better at growing antirrhinums and mimosas.

Judge-3: Ah a horticulturalist?
Do-Much-More: Well I don't know how much a horticulturalist weighs, but I reckon the average horticulturalist must be worth twice that much in gold.




Do-Much-More and the Next Quantum Leap

The chatbot's name, Do-Much-More, owes its origins to the first well-known chatbot program 'Eliza', which was developed by Professor Joseph Weizenbaum at MIT in the early 1960s. Eliza was named after Eliza Dolittle in Shaw's play Pygmalion. David Levy called his first chatbot program Do-A-Lot (in contrast to 'Dolittle'), and when enhancing Do-A-Lot to create the current generation of the program he changed the name to Do-Much-More for obvious reasons.

The prize winning version of Do-Much-More is designed to respond with generalities, moving a conversation along in a lighthearted way without dealing with any details of a specific topic. As can be seen from the above excerpts from the 2009 Loebner Prize competition, it does this reasonably well.

The team at Intelligent Toys Ltd has already started on the next quantum leap in performance of their chatbot, employing a novel technique devised by Dr. Levy that will enable Do-Much-More to converse on any specified subject. Levy is convinced that there is a big market for such chatbots on corporate web sites. He says: 'Companies will find it very appealing when visitors to their web site can carry on conversations for as long as they wish about the company and its products.' Levy also sees a huge potential for the next generation of Do-Much-More for entertainment web sites: 'Imagine being able to chat to a virtual persona who is an Italian food freak, an avid Manchester United supporter or an expert on butterflies, . . . whatever subject you wish. The commercial potential is staggering.'

For more information please contact Dr. David Levy at Intelligent Toys Ltd.