[Don Patrick]

Ideally for me, if the AISB can provide a version of the websockets LPP that already ties into the CoCoHub API, I could just piggyback on that, or at the very least use their code as example. Since an online competition makes it possible to set up and monitor the interface myself, the added complexity is not a big problem.

We've discussed the potential of online cheating at length in the past. In summary: The likelihood of cheating is reduced when there is nothing at stake (I would prefer that there is), when the goal is not to imitate a human, or when running the test at ungodly hours, lengths, and/or unpredictable intervals. There will always be a possibility of one or more people tag-teaming with a chatbot, but in the event of uncanny performances one could also stipulate that the winning chatbot makers provide a plausible explanation of their methods to the organisers. The Winograd Schema Challenge required that winners share their code and algorithm in a paper, and while I would not agree to go that far, a less formal explanation could suffice if needed. The quirks of GPT for instance are fairly distinguishable. Aside from that, most of us are familiar with the various workings of chatbots, and I doubt many would want to win by cheating and being found out after the resulting scrutiny.

[MikeB]

One way to prevent cheating might be to run a speed test afterwards where the bot is flooded with messages, and it must be determined to respond faster than a human can (with some detail, not just saying "ok" "dont know").

If you have a delay to simulate typing, then also having an option to turn that on/off.

[squarebear]

I agree with Mike and  there's no reason why a bot can't respond pretty much instantly. If it's voice based, it should also display a transcipt of its response, to minimise the risk of a human in the middle. Anyone could pretend to talk like a bot using audio but not many will be able to type a 10-20 word sentence in a second.

[Don Patrick]

It's a good idea in theory, but not all chatbots have the speed of rule-based lookup tables. Implementations of GPT are slow, Facebook's Blenderbot is slow, my program's reasoning processes are slow, and internet connections themselves can lag.

[Denis ROBERT]

One way to prevent cheating might be to run a speed test

Or perhaps something like that :

http://vixia.fr/images/Iamarobot.jpg