Ai Dreams Forum
Artificial Intelligence => AI News => Topic started by: squarebear on April 28, 2019, 10:05:43 pm
-
The AISB have posted details about this year’s Loebner Prize event.
http://www.aisb.org.uk/events/loebner-prize
By coincidence, I was at the same expo in London as one the organisers and have clarified a couple of things.
1 - Anyone who wants to enter the main prize still has to provide a human like bot. This must be standalone (no internet access allowed.
2 - You don’t have to use the prohibitive Loebner Prize protocol, as your bot will not be competing against another human.
The main reason I'm posting this here is that the Loebner Prize has dramatically changed since Hugh's death and is now part of a larger event where pretty much anything AI related goes. So all you guys working on robots or non conversational AI can take part in the event. The main prize is still for a conversational AI but at least now other aspects of AI and robotics can be featured.
-
Thanks Steve. Jess needs a lot more work, but maybe one day :)
-
1 - Anyone who wants to enter the main prize still has to provide a human like bot. This must be standalone (no internet access allowed.
The main reason I'm posting this here is that the Loebner Prize has dramatically changed since Hugh's death and is now part of a larger event where pretty much anything AI related goes. So all you guys working on robots or non conversational AI can take part in the event. The main prize is still for a conversational AI but at least now other aspects of AI and robotics can be featured.
Siseneg is a human-like bot and I can't enter.
-
Closing date is this Friday.
http://aisb.org.uk/events/loebner-prize
Ruebot - You can still have your bot featured in the main exhibition. There's no restrictions in that part of the event.
-
Steve,
The following has the word, Evaluation, which is a bit ambiguous. Please explain if you're able.
If your chatbot, and any associated media, is already available to be sent for evaluation, please attempt to upload it at the following link.
Do you know what they mean by: 5. Do you give permission for your chatbot to be evaluated by a panel of judges, such that they may award prizes?
Does that mean answering 20 questions or having the software examined?
Thanks!
-
Good question. The format of the Loebner Prize has changed quite dramatically this year. There is no longer a qualifying round of 20 questions or just 4 entries and the Turing Test part of it has been removed. Instead, a panel of judges plus members of the public evaluate each invited entry to determine its "human-ness", presumably by typing to each one and rating the answers the chatbot gives.
-
Thanks Steve.
Of course, each entered Chatbot would/will receive the same, exact questions as all the other chatbots, correct?
Will or has the Grading changed? Is it still something like 3 - for best, correct answer, 2 - for a good answer but partly correct, 1 - for a pretty good guess or near miss and 0 for either no attempt or a "not even close" answer?
Just curious and I do hope they ask some better-worded questions than they did last year?
Some questions trampled the English grammar yet they expect the chatbot to abide by them. Imagine that... ;)
Cheers!
-
The first mention of evaluation concerns the selection, i.e. whether the submissions meet minimal quality standards (nothing stringent).
The evaluation in question 5 I assumed to be separately taking place at the event in September because scores from the judges will be combined with scores from the visitors. When I asked for elaboration on the judging criteria, they strictly answered "The jury and public will be asked to vote on the “most human-like†chatbot for the Loebner Prize." So I take it this is what they will literally do: Visitors and judges will have to decide for themselves what "human-like" means to them and ask their own impromptu questions, as has always been the case in Loebner Prize finals.
-
Yes, the stage with qualifying questions and grading has gone. I understand it the same way as Don does and assume that each entry will be on a long line of desks where people (judges/members of the public) can talk to each of them and then score them on how humanlike they are. There's no details on how this scoring system will work but I assume it's either going to be voting each chatbot on a scale of say 1-10 or more likely, just picking out the bot they thought was best.
-
Appreciate your thoughts guys.
To me, it sort of sounds a lot like a soup sandwich in the making.
Should be interesting, to say the least.
Good luck to you both!