Ai Dreams Forum
Artificial Intelligence => AI News => Topic started by: squarebear on January 26, 2021, 08:29:08 pm
-
I was sent this by one of the members of the AISBX, which I said I would pass along.
"Basically, CoCoHub wants to host AISBX (incorporating The Loebner Prize) this year.
For anyone who can make a chatbot that conforms to LPP (the 'modern' websockets version) or CoCoHub's API, they can provide an email address for that chatbot. So when you're in any zoom meeting you can invite that email address, and you'll be joined in the meeting with a 3D, voice-driven, avatar representing your chatbot.
Maybe bots that can't interface with CoCoHub can be 'exhibited' anyway, just with an external link, rather than something more integrated."
It might be cool for some botmasters to see their work in 3D. The only bad side is that the LPP is back but it's either this or nothing, as only CoCoHub have the resources and are willing to do this.
What do you think?
-
Coco / AI Conversation Systems seem to be well funded to host the Loebner Prize well into the future...
They need to keep some other websocket version, or they're just edging out their competition... Can't imagine creating anything with CoCoHub that has deep complexity and nuance. It's supposed to be a mad scientists competition IMO.
-
I'd want to know there is prize money and that it would be held in the fall to give time to sync to their world.
-
I'd want to know there is prize money and that it would be held in the fall to give time to sync to their world.
Good to hear from you Bruce - I'll pass thet back to the origanisers and post here if I hear any more.
-
This discussion is based on early feedback that is subject to change.
As such, it is for discussion purposes only. Do not rely on it otherwise,
especially for the Loebner Prize 2021
I saw these channels: Alexa, Embedded, Facebook, Telegram, Phone, Zoom
Embedded seemed the simplest one. Other channels may need software which
normally takes a little more time. The CoCoHub setup part was easy and you
get to see what the 3D avatars look like.
These are some first impressions.
-
If you can't connect peoples mad scientist projects via a universal protocol then the Loebner becomes a CoCoHub hobbyist "my first chatbot" competition...
-
Hi, Squarebear.
FYI, I now live in the UK in Alcester. Where do you live?
-
If you can't connect peoples mad scientist projects via a universal protocol then the Loebner becomes a CoCoHub hobbyist "my first chatbot" competition...
As stated, in previous years (politely and friendly), maybe that is not such a bad idea.
Building a hobbyist "my first chatbot" is not trivial, I think. ( No offense, Mike, just saying.)
Also, a good, reliable sponsor, is really needed. And, it seems CoCoHub is really in demand
with the way things are.
-
Have they removed the long present rule that bots have to be run locally and can't connect to any servers through the internet? I really think a GPT-3 based bot could win this thing, but no way that can be run locally.
-
Have they removed the long present rule that bots have to be run locally and can't connect to any servers through the internet? I really think a GPT-3 based bot could win this thing, but no way that can be run locally.
GPT-2 runs locally, I ran it on my home PC, it's 95% as good as GPT-3, except for extra abilities they gave it though, which may be a real bummer if it has Persona Dialog and sentence finishing and a chatlog dataset it trained on.
-
Interesting thing actually: I heard GPT3's terms of use don't allow people to use it for making chatbots with. I think it would stand a good chance of sounding like a weird human, until the judges start asking questions that involve numbers or logic.
I don't believe the intention is to build a chatbot from scratch on CoCoHub, just to use API calls to retrieve and send the input and output.
-
GPT-2 runs locally, I ran it on my home PC, it's 95% as good as GPT-3, except for extra abilities they gave it though, which may be a real bummer if it has Persona Dialog and sentence finishing and a chatlog dataset it trained on.
The GPT-2 model has 1.5 billion parameters while GPT-3 has 175 billion parameters, that's a factor over 100x and the quality of output is equally an order magnitude better under GPT-3 than GPT-2. The computer requirements needed to run it is also an order of magnitude bigger, needing close to $200,000 of computer hardware to run the pretrained model. It's exclusively licensed to Microsoft, so you couldn't install it if you could afford the machine to run it. Right now has to be an API call to the OpenAI/Microsoft servers.
Interesting thing actually: I heard GPT3's terms of use don't allow people to use it for making chatbots with. I think it would stand a good chance of sounding like a weird human, until the judges start asking questions that involve numbers or logic.
I don't believe the intention is to build a chatbot from scratch on CoCoHub, just to use API calls to retrieve and send the input and output.
OpenAI has an approval process requiring you show how you are dealing with the various dangers they have identified in GPT-3 like its potential bias and potential to spew divisive toxic language among other requirements. I have a bot with "conditional approval" from OpenAI in beta testing now, you can read about it here https://aidreams.co.uk/forum/ultrahal/ultra-hal-7-5-beta-available-powered-by-openai-gpt-3/
It's a hybrid of the Ultra Hal chatbot and GPT-3, so not pure GPT-3. Ultra Hal can do simple math without involving GPT-3 and the scripting of Ultra Hal tries to keep GPT-3 to a consistent friendly personality, has some content-filtering, and give it some simple learning ability in the form of long term and short term memory.
So as long as the Loebner 2021 rules would allow for API calls to Zabaware and OpenAI servers over the internet, I'd love to enter it.
-
GPT-2 runs locally, I ran it on my home PC, it's 95% as good as GPT-3, except for extra abilities they gave it though, which may be a real bummer if it has Persona Dialog and sentence finishing and a chatlog dataset it trained on.
The GPT-2 model has 1.5 billion parameters while GPT-3 has 175 billion parameters, that's a factor over 100x and the quality of output is equally an order magnitude better under GPT-3 than GPT-2. The computer requirements needed to run it is also an order of magnitude bigger, needing close to $200,000 of computer hardware to run the pretrained model. It's exclusively licensed to Microsoft, so you couldn't install it if you could afford the machine to run it. Right now has to be an API call to the OpenAI/Microsoft servers.
Interesting thing actually: I heard GPT3's terms of use don't allow people to use it for making chatbots with. I think it would stand a good chance of sounding like a weird human, until the judges start asking questions that involve numbers or logic.
I don't believe the intention is to build a chatbot from scratch on CoCoHub, just to use API calls to retrieve and send the input and output.
OpenAI has an approval process requiring you show how you are dealing with the various dangers they have identified in GPT-3 like its potential bias and potential to spew divisive toxic language among other requirements. I have a bot with "conditional approval" from OpenAI in beta testing now, you can read about it here https://aidreams.co.uk/forum/ultrahal/ultra-hal-7-5-beta-available-powered-by-openai-gpt-3/
It's a hybrid of the Ultra Hal chatbot and GPT-3, so not pure GPT-3. Ultra Hal can do simple math without involving GPT-3 and the scripting of Ultra Hal tries to keep GPT-3 to a consistent friendly personality, has some content-filtering, and give it some simple learning ability in the form of long term and short term memory.
So as long as the Loebner 2021 rules would allow for API calls to Zabaware and OpenAI servers over the internet, I'd love to enter it.
GPT-2 has 4 model sizes, the later 3 run on household PCs and feel 90% as accurate. Especially 3rd model size.
-
Hi, Squarebear.
FYI, I now live in the UK in Alcester. Where do you live?
Hi Bruce. I'm up here in sunny Yorkshire.
-
So as long as the Loebner 2021 rules would allow for API calls to Zabaware and OpenAI servers over the internet, I'd love to enter it.
The competition is all totally online this year. Nothing local, so any API calls you need to make are fine, assuming CoCo can support it. There is the option where you can provide a link to an external resource for users to talk to your bots, which is something I'll probably do as we already have a working 3D avatar.
-
Ideally for me, if the AISB can provide a version of the websockets LPP that already ties into the CoCoHub API, I could just piggyback on that, or at the very least use their code as example. Since an online competition makes it possible to set up and monitor the interface myself, the added complexity is not a big problem.
We've discussed the potential of online cheating at length in the past. In summary: The likelihood of cheating is reduced when there is nothing at stake (I would prefer that there is), when the goal is not to imitate a human, or when running the test at ungodly hours, lengths, and/or unpredictable intervals. There will always be a possibility of one or more people tag-teaming with a chatbot, but in the event of uncanny performances one could also stipulate that the winning chatbot makers provide a plausible explanation of their methods to the organisers. The Winograd Schema Challenge required that winners share their code and algorithm in a paper, and while I would not agree to go that far, a less formal explanation could suffice if needed. The quirks of GPT for instance are fairly distinguishable. Aside from that, most of us are familiar with the various workings of chatbots, and I doubt many would want to win by cheating and being found out after the resulting scrutiny.
-
One way to prevent cheating might be to run a speed test afterwards where the bot is flooded with messages, and it must be determined to respond faster than a human can (with some detail, not just saying "ok" "dont know").
If you have a delay to simulate typing, then also having an option to turn that on/off.
-
I agree with Mike and there's no reason why a bot can't respond pretty much instantly. If it's voice based, it should also display a transcipt of its response, to minimise the risk of a human in the middle. Anyone could pretend to talk like a bot using audio but not many will be able to type a 10-20 word sentence in a second.
-
It's a good idea in theory, but not all chatbots have the speed of rule-based lookup tables. Implementations of GPT are slow, Facebook's Blenderbot is slow, my program's reasoning processes are slow, and internet connections themselves can lag.
-
One way to prevent cheating might be to run a speed test
Or perhaps something like that :
(https://aidreams.co.uk/forum/proxy.php?request=http%3A%2F%2Fvixia.fr%2Fimages%2FIamarobot.jpg&hash=fdaac46ff724c39527baef0dc324fd9e623cb80b)
http://vixia.fr/images/Iamarobot.jpg (http://vixia.fr/images/Iamarobot.jpg)
;)