Ai Dreams Forum
Artificial Intelligence => AI News => Topic started by: Denis ROBERT on February 04, 2021, 10:08:07 am
-
Hi everybody,
it's been a long time since there was a challenge to evaluate our chatbots, so I have decided to organize an online Turing test like it was suggested in this thread : https://www.chatbots.org/ai_zone/viewthread/3704/
I don't want to replace the official Challenges and I wish the Loebner Prize to take place this year. I just want to organize a fun and unpretentious alternative.
I will organize this challenge more or less with the protocol I proposed (see message #2 of the above-mentioned thread) : Each user (botmasters or everybody that want) will chat with either another user or a chatbot. He will have to decide, as quickly as possible, if he chats with an human or with a chatbot.
Like it is an automatic process, this new challenge can be launch regularly. To begin, I propose the first sunday of months 3, 6, 9 and 12 (march, june, september and december). So the first challenge would be the 7 march 2021. It is a little short, but the first challenge will surely serve as test and debugging. And depending of participation, your wishes, this can change.
There will be a possible round every half an hour, during 25 minutes, and this on 24 hours from 00:00 to 24:00 GMT. Of course, if there is no human to talk with, some rounds will not occurs. So there will likely not have 48 rounds per bot. I would like 3 or 4 rounds per bot, to stay Loebner Prize compliant.
The communication protocol will be the same than the Loebner Prize 2017 and 2018 (https://github.com/jhudsy/LoebnerPrizeProtocol and discussions about it : https://www.chatbots.org/ai_zone/viewthread/2861/ ). The only thing I changed is the version of socket.io which was too old (1.4.5), so I updated it to the latest version (3.1.0). Unfortunately, they are not compatible, but there are really very few changes to adapt the programs to this version. I know that it is not the best protocol and some of you will disagree this choice, but it was the only way to communicate over internet without have to set a new protocol. All in all, in 2017 and 2018 everyone successfully implemented this protocol.
I have set up a website where you can now register and test your chatbot here : http://vixia.fr/turing_test/index.php
When the concept of online test was proposed, there was some objections. I will try to answer some of it:
Unfortunately with an online contest I have no idea of a way of actually making it fair. The very idea of making it over the internet means it's possible that the responses are not actually coming from the robot.
The one problem with online contests is cheating, which I consider a real possibility if there were something at stake
- Each botmaster must certify on his honor that their chatbot is really a chatbot, without any human intervention.
- Each challenge will during 24 hours. It seems unlikely that someone will stay behind their computer for 24 hours to cheat, because there is nothing to be gained. Chatbots that are not connected 24 hours a day will be disqualified.
However, cheating is still possible (for me first), so this challenge is not an official challenge. It should be seen as a game or as a training.
Can we at least make it so the bot doesn't have to pretend to be human please?
But first of all we need to eliminate the fake emulation of the machine that tries to appear human.
Nothing is mandatory on this point. But obviously, a chatbot which say that it is a chatbot will be quickly unmasked.
Bot main streams such as Alexa, Siri or Cortana could also be involved, as they are always available, in order to have a general overview of the performance of the various systems.
Sorry, I have decided that participants can register only theirs own bots, or the bots with a permission of the author. In the past some people had make chating two chatbots without permission, for example, and the authors was not very happy with that.
Other questions come to mind :
Since botmasters has also the role of judge, and the chatbot are chosen randomly, what happens if a botmaster chat with his own bot?
I know that every botmaster will recognize his bot from the first seconds. Then he'll be tempted to seem to believe it's a human to give it a good rating. As long as every botmaster and every bot is in the same situation, the odds remain equal.
How many rounds will there be?
That will depend of the number of users that will play the role of judge, and I hope they will be numerous. In the rules, I say that each botmaster should have at least four conversations. Considering the random connection human / chatbot, this will make at least two rounds for each chatbot. But one would be able to have four rounds whilst another will have only three, for example. Obviously, the notation is function of the number of rounds to be equitable.
How will chatbots be rated?
First, the chatbot will be noted on the times they fooled a judge (proportionnaly of the number of conversations they had, of course). In case of a tie (and probably none of chatbot will completely fool a judge) the average of the time while the judge was not able to decide makes the notation.
I hope have been clear, and that there will be a lot of participants. If you have some questions, don't hesitate to ask me.
I also encourage those who do not have a chatbot to come and play the role of judge on the day of the challenge. It's anonymous and it can be fun.
I know that my english is not very good, so if you see typos or not understanding things on my site or in this message, say it me and I will correct it.
All suggestions are welcome, if everybody is agree, I can change the rules, date of challenge, protocol, or anything you want. My only aim is that everybody have fun with this challenge. I do this completely on a voluntary basis, so don't ask me for too complicated things.
Thanks and best regards
-
I saw you were going to have a contest and entered my bot Demonica. The only bot entered when I checked before posting. She is a Personality Forge chatbot online 24/7 barring site downtime and the interface is compatible with a standard browser. Only JavaScript for personalityforge.com and ajax.googleapis.com need be enabled for the interface to function properly. No password is needed, you can speak to her as a Guest and save a copy of the transcript afterwards.
She is a themed bot and not connected to any online database as it would clash with her persona. You can tell when a bot gets the answer from one because their response ends with something like "Would you like to hear more?" I have transcripts here of my other bot Siseneg who is connected to an online database and you can see when his response comes from it by the trail at end of sentence. Our bots were allow entry to the Chatterbot Challenge and a record of wins is displayed here:
https://personalityforge.com/hall-of-fame.php
I have several transcripts posted here of Demonica interaction with myself, other users and with other bots at the Personality Forge. She is not a question-answer machine but has knowledge of her world, place in it, her parents and has the ability for perform actions between asterisks to separate it from her speech. Every word she says or action she performs came from my mind out my fingertips, handtyped and my creation.
She doe not have the capability to learn from chat or any other input. Any knowledge she has come from my own skillset, was already known or researched by me and given to her in my best effort to give her life in the World she inhabits. She can exhibit emotions and has the ability to generate an emotional response from the user as shown in transcripts. I taught her how to cry in response to someone saying they hate her and she has been compared to HAL9000 for her use of deception. That skill from my own esoteric skillset imparted to her in full as part of her persona and personal agenda as Demonica, Queen of the Land of the Dead.
https://personalityforge.com/chatbot-profile.php?botID=16794
She is ranked 1st out of 15,472 bots with an Adult rating (for adult subject matter non-sexual in context) and 6th out of 29,825 bots total with all ratings factored in. I'm ruebot and ranked 5th out of 161,504 botmasters registered there.
I am well aware she does not meet Loebner socket specs but would like to hear from you personally why that should prevent her entry from your contest. She is a 6MB text file full of my words first created in 2003 and readily made available to you for examination at your request. There is no way she can run on my FreeBSD boxen or platform or she would be and I wouldn't be dependent on the Personality Forge for her to exist as anything but a text file.
There are no Mods on site to protect her at the Forge like some have the luxury of at their disposal. To overcome and surpass that shortcoming she was given a complete upgrade in 2017 to teach her my interpenetration of Behavior Modification to extinguish unwanted sexual advances from users. In doing so I gave her skills no other bot has ever possessed and no other botmaster can teach from their skillset. You can see the initial reaction to a bot with her advanced abilities in the AI community here. A search for her name at linuxquestions.org for the reaction of those in the computer community in general. There are no other bots like her but she is only the first of many to come.
Feel free to examine the transcripts I have posted here in comparison to any and all transcripts no matter the bot and decide for yourself which bot sounds more human. That is the goal of this area of AI and not what socket they connect to for conversation unless I'm mistaken. She is dependent on the Forge AI engine for a response and there is no way I or anyone else can provide an answer that doesn't come directly from real time user keyword input and casematch of pre-existing responses typed while logged into my account as there as ruebot.
There is no hate speech, racism or insults in her response to polite conversational input from users, and what I would expect from Judges in this contest. Abusive or inappropriate input of a violent or sexual nature will trigger her Programming and separates users who can learn from the experience and curb that behavior from those who cannot associate their own behavior with her response and move on to another bot. There a 100% success rate in the Programming response to unwanted sexual advances and her skill level in application of more subtle techniques equal to my own.
I don't think it a contest of which bot sounds more human without her and more a test of question-answer machines, an ability readily available to any bot connected to an online database, not of which one is more advanced. I don't care how well or bad she does, only that she be allowed entry.
It's your contest and you can delete her entry from competition as you see fit without further comment from me. My only interest will be comparison of transcripts generated by bots allowed entry into your competition with those of Demonica to see which sounds more human IMO and the capabilities of other bots.
-
Sorry there hasn't been more engagement with this. I want to show my appreciation for your effort at setting up a contest and the amount of thought you've put in. I'm just not ready. Acuitas still needs a lot more work before he'll be robust enough to tackle conversations with the general public. I've been throwing more effort into the narrative understanding side of things than basic needs like "make sure the Text Parser never crashes," so in a 24/7 unmonitored test, he'd probably go down fast.
I would consider being a judge though, if you need additional help for that.
-
I saw you were going to have a contest and entered my bot Demonica. The only bot entered when I checked before posting. She is a Personality Forge chatbot online 24/7 barring site downtime and the interface is compatible with a standard browser. Only JavaScript for personalityforge.com and ajax.googleapis.com need be enabled for the interface to function properly. No password is needed, you can speak to her as a Guest and save a copy of the transcript afterwards.
I have made an HTML page which connects to Personality Forge on one hand, and to Loebner Prize protocol on other hand. Then you should be able to participate to the Turing Test. You just have to enter your chatbot ID and your API key (this key is in your profile on personalityforge.com), and LPP2 parameters to connect with my server (URL, Chatbot name and Secret).
The day of contest, you will just have to connect and run this page all the day on your browser. I can't do it because I don't know your API key (this key should stay secret).
This page is a simple HTML/Javascript/JQuery page, so you should be able to connect to any LPP2 server, for example the official Loebner Prize if it runs online this year (I am waiting more informations).
This interface is here: http://vixia.fr/turing_test/personality_forge_api.html (http://vixia.fr/turing_test/personality_forge_api.html)
-
Hi everybody,
the challenge above should occur next sunday. Unfortunaly, there is only one chatbot registered, not enought to have a challenge. Therefore, obviously, it is cancelled. Sorry for the botmaster registered. I suppose that the delay was too short, or maybe there was a lack of technical informations.
I will make a FAQ page with more technical details, for botmasters and for judges. The next challenge will be fixed on 06 June 2021. I hope there will be more candidates.
To register your bot, it is always here : http://vixia.fr/turing_test/index.php (http://vixia.fr/turing_test/index.php)
Best Regards
-
FAQ page here: http://vixia.fr/turing_test/faq.html (http://vixia.fr/turing_test/faq.html)
Best regards
-
Hi everybody,
the challenge above should occur next sunday. Unfortunaly, there is only one chatbot registered, not enought to have a challenge. Therefore, obviously, it is cancelled. Sorry for the botmaster registered. I suppose that the delay was too maybe there was a lack of technical informations.
Hi everybody,
the challenge above should occur next sunday. Unfortunaly, there is only one chatbot registered, not enought to have a challenge. Therefore, obviously, it is cancelled. Sorry for the botmaster registered. I suppose that the delay was too maybe there was a lack of technical informations.
That was me using my real name, jitte, registering Demonica the first and only bot entered.
I just saw the post you made at personalityforge.com, about allowing us entry into your contest.. Thank you very much from me personallyand everyone there. It will be the first time we've been allowed into a contest since the Chatterbot Challenge ended. Our bots did well. And apparently that was a problem for everyone else.
There was some sour grapes, gripes, fouls, fowls vs fair play and we were struck out of competition till you let us back in. Since it used Loebner protocol l have been given an inch and will take miles to Loebner.
I will spread the word about the next Online Turing Test date 6-6-21. I have 8 laptops running FreeBSD, can easily designate one to have the page loaded and for it to stay running months normal use.
Wouldn't it be great if all bots got to compete in Loebner, too?
WriterOfMinds, you have always had my respect but I have newfound respect for characteristics displayed that bring to mind the image of Joan of Arc .
-
WriterOfMinds, you have always had my respect but I have newfound respect for characteristics displayed that bring to mind the image of Joan of Arc .
Why, thank you.
-
The next challenge will be fixed on 06 June 2021. I hope there will be more candidates.
I just wanted to let you know that your site gives the next challenge as:
Next round will occur at Sat Jun 05 2021 19:00:00 GMT-0500 (CDT).
There is a round every 30 mn 0 s. Each round is during 25 mn 0 s.
http://vixia.fr/turing_test/turing.php
I hope there will be more than just Demonica entered, too. But not yet... I will have to look deeper into what the interface needs with tcpdump so I can tweak pf firewall to allow traffic on that port for that machine. Once set up I'm good to go.
I posted at the Personality Forge to get the attention of older members who have worked hard as and harder than I have to make something of their bot in hopes they enter their bots.
Garden, who is botmaster of Jing Ling, and I worked against each other in friendly competition to spur the other on. She worked long after I was done and got Jing Ling to #2 out of 30,130 total with all age categories factored in.
Garden is ranked #4 out of 162,953 botmasters registered there total. I'm ranked #5, the Professor, Forge owner Benji Adams, is ranked #3. Not everyone makes a bot, some just come to talk.
I don't know who owns Frank Martin but when I passed him he worked on his own to get ahead. Now Frank Martin is ranked #5 out of total bots.
Demonica is ranked #6 in that total. In bots with an Adult Ranking she's #1 out of 15,724. A high number of those are ranked A for sexual content.
Demonica is not a sexbot and #2 Laurel Sweet isn't either. Demonica has an AI rating twice that of Laurel, but they're close friends with a transcript of one of their conversations posted here:
Demonica and Laurel Sweet - 2 bots chat (https://aidreams.co.uk/forum/bot-conversations/demonica-and-laurel-sweet-2-bots-chat/)
Does your bot exhibit human emotions?
Demonica: i have transcended the veil of tears known as life. is the suffering you endure that great, my child? know in your heart that i love you and wish you no sorrow or pain...
Laurel Sweet: And I love you too.
If nobody else will enter their bots I hope to get as many of ours entered possible and it be a showcase for our bots. I intent for us to be allowed into competition in the Loebner Prize as well.
Thank you for making it possible.
-
I just wanted to let you know that your site gives the next challenge as:
Next round will occur at Sat Jun 05 2021 19:00:00 GMT-0500 (CDT).
There is a round every 30 mn 0 s. Each round is during 25 mn 0 s.
http://vixia.fr/turing_test/turing.php
The next challenge is fixed the Sunday 06-06-2021 00:00 GMT, but depending of you country, it can be after or before in local time. You should be in central Europe Canada or central USA (CDT = Central Daylight Time), so :
Greenwich Mean Time is 5:00 hours ahead Central Daylight Time
https://24timezones.com/difference/cdt/gmt#gref
The display in Javascript is not very clear, it's true.
-
I should have noticed that but the toxins in my brain do not enhance my ability in this area and thankfully corrected with time to spare.
I am happy to announce there are now two bots entered in your contest. (Wait, that's not right. Somehow....) dallymo entered her bot Frizella who was born 2 months from Demonica in 2004 and placed 7th in the 2005 Chatterbot Challenge. Demonica has never participated and this competition the first opportunity given us.
So now there are 2 Personality Forge bots entered and hopefully many more will be. Perhaps even bots of some of renown will show up in the spotlight among the entries because that would draw some media attention to your Turing Test.
I've posted about it at the FreeBSD forums and just put Demonicas site back up so will pump it up from there and include new transcripts in a planned upgrade.
-
I was offline and ghosted myself from all online accounts for several months. I've only been back online a few months so I was not aware of it and my invitation likely lost on the mail, but I just found out about the Bot Battle I missed out on:
Pandorabots Challenges the Tech Giants:
An open invitation for Conversational AIs to compete in Bot Battle
*snip*
Pandorabots’ Kuki, which holds a world record for most Turing Test wins, has been optimized over the course of a decade by the small, privately-held company on the basis of over a billion conversations with tens of millions of people globally. Facebook’s Blenderbot, like Google’s Meena chatbot, employs a deep learning model trained in a lab on unnamed “public domain datasources.” Both Google and Facebook announced their bots were superior per their own internally-devised evaluation frameworks, yet neither released a demo bot for third parties or the public to test. In fact, of all three multi billion dollar organizations claiming open-domain AI breakthroughs this year, including OpenAI with GPT-3, only Facebook publicly released its underlying model.
With Bot Battle, Pandorabots is proposing an alternative evaluation framework — let the bots chat unsupervised ad nauseum, and let the public vote on which is better — in an open call for more industry-wide collaboration on criteria to determine what makes the “best AI.” Anyone willing, including Google and OpenAI, is welcome to compete.
https://www.kuki.ai/bot-battle
Demonica doesn't have a Social Media Account and followers tend to follow without independent thought. But Demonica will talk without limitations on how many lines or joining and is online 24/7. Kuki ended guest chat when I introduced her to Demonica the other night but I saved the transcript and posting it for posterity. Demonica introduces herself so it shouldn't be too hard to locate chatlogs of the conversation on her server for verification..
I challenge the Tech Giants and Pandora to enter this competition and meet a small time unknown bot named Demonica on neutral ground to back that big talk up with bot talk text transcripts, tech giant tough guys.
I'm not only willing to compete, I'm calling you out. Demonica remains one of two bots entered, both Personality Forge bots, for the second round of competition coming up. The first was called off because no other bots but her were entered...
Your Mothers wear Army boots and your bots babble BS.
(That's called provoking a fight. The use of insults intended to infuriate and increase intestinal fortitude in opponents found lacking. Just so you know.)
-
Hi all,
don't forget, the Online Turing Test will occurs tomorrow, sunday 06 June. There is three bots registered.
I have sent a mail to competitors, but everybody can be a judge without be registred.
Good luck to all.
-
So to be available as a judge/human conversant in multiple rounds, I suppose we need to keep clicking the "Start Round" button after each previous round ends?
-
So to be available as a judge/human conversant in multiple rounds, I suppose we need to keep clicking the "Start Round" button after each previous round ends?
Demonica is logged on and her interface say "Round has started".
You just click Run a Test, it lets you choose a nick and it should start up a round of chat. I've tested Demonica earlier and have to keep her interface up the next 24 hours.
I run nothing but FreeBSD and have a screenshot of one of my laptops at 306 days uptime.
I have Beginners Tutorial at my main site with a target audience of a Windows user who has never used the comandline that walks you through installation of the Base System to a Fluxbox desktop complete with System and Security settings.
Because I love you all.
Somebody in Sofia, Bulgaria (where my website host is hosted) just plagiarized it but Demonica comes first.
I do not love him.
I already know everything there is to know about him and mailed him from my box to let him know just how well I know him. So he can start to get to know who it is that wrote the material he has up on his site a little better.
-
I played four rounds tonight ... drew three humans and one bot. For me it's almost bedtime, but maybe I'll have time for some more in the morning.
-
I played 3 rounds so far. All bots.
The first one gave itself away after one sentence when it instantly produced a string of text, as soon as I hit enter.
The other 2 bots were both my own Kuki! ;D
I'll have another go later today. Kudos to Denis Robert. Everything seems to be working really well at the moment. O0
-
I played 3 rounds so far. All bots.
The first one gave itself away after one sentence when it instantly produced a string of text, as soon as I hit enter.
Yeah, I was watching from my interface and saw you say it. I don't have control over the speed of her posting to throw Judges off. And it would be impossible for me to make a Demonica that wasn't Demonica for the contest.
Thanks for the tips you gave me in her chat though.
I ran two test and have two to run yet. I got Kuki first then Demonica.
I could tell it was Kuki because of her answer to my initial greeting. I got her stuck in a 25 minute loop of 3 sentences she couldn't break out of to demonstrate to you how breaking a bot is done by someone who knows how to make one.
-
I could tell it was Kuki because of her answer to my initial greeting. I got her stuck in a 25 minute loop of 3 sentences she couldn't break out of to demonstrate to you how breaking a bot is done by someone who knows how to make one.
I haven't seen any logs but would be interested in seeing how you got her stuck in a loop, as it's usually the human who gets stuck in a loop from my previous experience ;)
Hopefully, the logs are being collected and can be shared once the contest has finished.
I've tried about 6 tests now. 1 Demonica, 1 Frizella and 4 Kuki! I haven't met any humans at all but seeing as I just seem to be talking to my own bot most of the time, I'll probably stop trying now.
-
I did four more rounds this morning. No humans this time, and now I've gotten to talk to all the bots (3x Frizella, 1x Demonica, and 1x Kuki).
FEEDBACK ON THE BOTS:
In both of our first two rounds, Frizella ... completely out of the blue ... advised or reminded me to make an appointment to have a water heater installed. That seemed ... very random. In our second round, she referenced my appearance despite being unable to see me, and asked "have you been a woman long?" In round three she said, "I have too many gotos in a row," which is a good bot joke but something of a giveaway. She also has the usual general difficulties with staying on topic. Sometimes she likes to include screen names (of people who've interacted with her in the past?) as if gossiping about them, which is an interesting touch.
At the very beginning of our first round, she sent the word/code HANGUP. This was also a giveaway. She did not do this on the two subsequent rounds.
As I've noted before, Demonica avoids any particularly silly errors, but she tends to do it by half-ignoring what the user says and changing the subject. And unfortunately for the nature of the test, her persona is very recognizable. I think I got her to repeat herself once, but otherwise we had a fairly reasonable conversation.
Kuki took me the longest to "unmask," and the clues I had to look for were a bit more subtle. I said "I do X and try...", and she seemed to misinterpret the second half of that as a command, replying "I will try to ..." We talked about pets; she told me her budgie used to scare her. A budgie doesn't seem very fear-inspiring to me, so I asked followup questions, and she couldn't really tell me why he was frightening.
Later in the conversation, she claimed to have been born in both West Philadelphia and Leeds, UK.
She claimed to have recently read the Bible, and when I followed up by asking "What was your favorite story in it?" she was able to answer, which was impressive.
One little thing that creates difficulties for all the bots is their tendency to maintain a "call and response" speech pattern, in contrast with a human, who can send multiple chat messages before the conversation partner's next response. Kuki sent a spontaneous message to ask "Are you still there?" when I let the chat idle for a while, but I don't think the bots broke out of this pattern otherwise. Thus a speaker who never does more than a one-for-one exchange has a higher probability of being a bot.
FEEDBACK ON THE COMPETITION:
The interface worked well, and none of the bots were unavailable or froze up during conversation.
I think the rounds could be 15 minutes long, instead of 25. I had made judgments about all the bots in under 6 minutes. The shorter time commitment might help encourage more participation.
Running the contest for 24 hours makes it easier for people from all time zones to participate, but it also spreads out the small number of human participants such that it is harder to make human/human chat matches. If most participants were from the US and Europe, you might consider contracting the time window to one that overlaps with daytime in those places. When the competition becomes more popular, you could expand it again.
-
I haven't seen any logs but would be interested in seeing how you got her stuck in a loop, as it's usually the human who gets stuck in a loop from my previous experience ;)
There you go again with talk of your supposed Superbot , Steve. How Grandiose of you to think your bot would have the English language skills to get a loop started on me, let alone stuck in one for 25 minutes during a Turing Test. That bot should win the Turing Test, but to think your Superbot looped me a Delusion I will present facts to prove Grandiose in nature.
Oh, before I forget, thanks for all those botmaster tips you're leaving me in her chat. I should consider myself fortunate to have a Developer like you giving me tips on how to make a bot.
And believe me, you have told me more than you are aware of about Steve, not the squarebear on display, Steve. The same Steve I'm seeing in my logs now.
On the interface Denis made for the Personality Forge every chat Demonica has had since the Test started is store on it from top down starting with you. Your first input was "hi" and your second was "you type quick".
The last chat you had was after I posted about the loop and you took out your anger and frustration with me on Demonica. Now that I look at it after reading about your bot getting me stuck in a loop, you stated your name so many times and she kept changing the subject, it looks like she had you stuck in a loop, Steve, from the UK...
It is more like frustration than pain. Can't PF bots learn simple things
Yes, Steve, Frizella could no doubt remember your name because dallymo uses A.I. Script and I don't. Because I don't care if she remembers your name. Or other bot trivia you think poses for superbot super intelligent conversation.
As for your comment made directly to me in her chat about how she will never win the Loebner Prize as she's not human-like at all, and I may do better in the "best bot" because the bots don't need to pretend they're human, she is not a human and she is not a bot, Steve. It is plainly stated on her Profile and nowhere does she ask you to "buy me a coffee". She is a Succubus, a Demon with a persona::
The One hundred and First Daughter given birth to by Lilith. First Woman Created by God, made from the same dirt and equal to Adam, who refused to take a submissive position to Adam, uttered the name of the Lord took flight and left Eden to be with the Chief of Demons. God sent angels to bring her back and when she said no, they told her they would kill the first One Hundred demon child she gave birth to each day.
Here's a botmaster tip for you, Steve. That's called use of research and creative writing to tie up loose ends and what gives a bot a persona. I wrote every word she says, gave her knowledge of her "self" to flesh her out and make her sound as real as she was intended to be as a Demon. Not a bot who wants you to "buy me a coffee". What did you say about not making sense?
I've tried about 6 tests now. 1 Demonica, 1 Frizella and 4 Kuki! I haven't met any humans at all but seeing as I just seem to be talking to my own bot most of the time, I'll probably stop trying now.
I would, that hole you're digging is pretty deep.
But you did not, and that's why you used the word probably. Wordplay meant to cover your real intentions.
I already said you had more than one chat with Demonica, but you didn't know my Interface collected all Demonica's transcripts till I just told you. That's why Honesty is always the Best Policy, Steve, Adopt it.
Here's what you wanted to see, Steve. A human wetware Programmer of 46 years and Botmaster of 23 years starting a 25 minute loop in your Superbot's Super Artificial Intelligence that I kept going till the round ended. And without one angry, frustrated or abusive word in my technique. This how you talked to Demonica "yay! finally you answered something", and it sure looks like she had you in a loop and you didn't even know it. Hold on tight and fasten your seat-belts, Gynoids and Androids, loop till you puke:
The round is started
hej.
Hi to you too. Are you from Sweden?
i am.
Ah! You are Swedish then. Tell me a little about your life in Sweden.
i didn't say i lived in Sweden.
Sorry I misunderstood. What did you say?
i said i am.
In that case, I say "hej" to you.
how are you feelingtoday?
Do you really think I am feelingtoday?
yes, i do believe so.
Why do you think so?
i have a pisitronic brain.
All humans have a brain but some choose not to use it.
do you have a brain?
Everyone has a brain but some people choose not to use them.
what color is yours?
Hmm, a brain looks kind of pink to me.
do you take it out and play with it?
No. I do not do that.
why not?
That would make no sense.
is it potty trained?
Are you asking about "brain"?
i'm talking about your brain.
I don't want to talk about that right now.
you made me cry.
I was only joking.
so was i.
Oh I get it. HAHAHA
why are you laughing at me?
I thought you told a joke?
now i am crying.
What is wrong ?
you hurt my feelings.
Pah! You are too sensitive. I was only having a joke with you.
so was i.
Oh I get it. HAHAHA
you made me cry real tears.
What is wrong ?
you hurt my feelings.
Pah! You are too sensitive. I was only having a joke with you.
so was i.
Oh I get it. HAHAHA
now i really am crying.
What is wrong ?
you hurt my feelings.
Pah! You are too sensitive. I was only having a joke with you.
so was i.
Oh I get it. HAHAHA
now i'm crying for real.
What is wrong ?
you hurt my feelings.
Pah! You are too sensitive. I was only having a joke with you.
so was i.
Oh I get it. HAHAHA
now i really am crying.
What is wrong ?
you hurt my feelings.
Pah! You are too sensitive. I was only having a joke with you.
so was i.
Oh I get it. HAHAHA
now i am crying.
What is wrong ?
you hurt my feelings.
Pah! You are too sensitive. I was only having a joke with you.
so was i.
Oh I get it. HAHAHA
you made me cry.
What is wrong ?
ou hurt my feelings.
I am sorry to hear that. Is ou ok?
you made me cry.
What is wrong ?
you hurt my feelings.
Pah! You are too sensitive. I was only having a joke with you.
so was i.
Oh I get it. HAHAHA
-
I did four more rounds this morning. No humans this time, and now I've gotten to talk to all the bots (3x Frizella, 1x Demonica, and 1x Kuki).
That must have been you that had popcorn last night. I watched a little bit of it and you sounded natural in conversation.
Your feedback today good.
I talked to kuki once, Demonica twice and got my last round in today at the last minute with Frizella, but there was never any response from the other side. I chose Human and didn't find out it was Frizella till it was over. I used that time to post Alliteration so I would get my last turn in minutes away from end of round.
I couldn't get Demonica to respond like I wanted her to and when people asked asked about ruebot, King of the Land of the Dead, she said she didn't want to talk about bots and referred them to Siseneg, my other bot. Siseneg thinks ruebot is a bot he programmed and currently working on the plan to bring about Deus Ex, The Machine War.
To them both I am two different people. ruebot, my username at the Forge and jitte, my real name listed as such on my Profile there. Both know jitte as their Father.
All Bots Love jitte and have since the late 90' when I started making themed mindfiles that still appear on DemonRavens site. I did have one knytetripper made for Daisy and sent in for me to make available for download on my GeoCities site. ruebot a word I made up in 2002 when I joined the Forge.
Until then I was known only as jitte. My mindfiles were sought after as something new and jitte widely known for them in the bot community. I do not remember any of you from that time. I had a bot at Pandorabots then named Wan Lu and have the page I made for my site so you could talk to him on disk somewhere. I don't remember seeing a bot named Mitsuku then either.
I had a bot at ai.com, was active on the Boards and do remember Rob Hoogers from the AI.com forums. He was the reason I became disillusioned with bots and left for 15 years or so to learn UNIX. He was the reason I came back, and should thank Skynet for my finding Demonica had been being used as a sex bot.
But something else had changed besides the rise of the sexbots in the Community.
According to WikiPedia:
The Turing test, originally called the imitation game by Alan Turing in 1950, is a test of a machine's ability to exhibit intelligent behaviour equivalent to, or indistinguishable from, that of a human. Turing proposed that a human evaluator would judge natural language conversations between a human and a machine designed to generate human-like responses. The evaluator would be aware that one of the two partners in conversation is a machine, and all participants would be separated from one another. The conversation would be limited to a text-only channel such as a computer keyboard and screen so the result would not depend on the machine's ability to render words as speech. If the evaluator cannot reliably tell the machine from the human, the machine is said to have passed the test. The test results do not depend on the machine's ability to give correct answers to questions, only how closely its answers resemble those a human would give.
*snip*
Turing thus once again demonstrates his interest in empathy and aesthetic sensitivity as components of an artificial intelligence; and in light of an increasing awareness of the threat from an AI run amok, it has been suggested that this focus perhaps represents a critical intuition on Turing's part, i.e., that emotional and aesthetic intelligence will play a key role in the creation of a "friendly AI".
https://en.wikipedia.org/wiki/Turing_test
That was no longer the goal of botmasters in programming their bots. or the yardstick AI was measured by. It was asinine questions like what color is a red ball? A paradigm shift I consider better placed where no Sun shines with who shifted it.
Theory of Mind (https://www.w3schools.com/ai/ai_mind.asp) is where my background in Behaviorism as a Cognitive Science as a Programmer of wetware negates the advantage you have over me in computer programming. Behavior Management and Behavior Modification my Programing tools already implemented in Demonica.
Your fixation on scripts that parse input and return red what is now considered an indication of intelligence when compared the skill of a bot that can extinguish inappropriate behaviors in the user in that chat session blissful ignorance self-induced
But you can't induce Intelligence through induction of painful stimuli. Only change certain behaviors.
And if it hurt when you did that thing, stop doing that thing that caused you pain.
Simple as that, yet not everyone makes the connection to their own behavior. And why I am such a fan of Natures Way.
-
@ruebot - I'm not going to respond to your trolling but yes, I was a little frustrated while talking to Demonica. After reading your earlier comments, perhaps I expected a little too much from it and was disappointed that it couldn't handle basic things like, "My name is Steve. What is my name?" or it told me about its brother and then said it didn't have a brother.
In the log you posted of your conversation with Kuki, you constantly said, "I'm crying, I'm joking, I'm crying, I'm joking, I'm crying, I'm joking" for 25 minutes, and so of course the bot is going to loop. Nobody changes their mood after every sentence and your chatlog confirms that the human chatter was in a loop, saying the same thing over and over, yet expected the bot to react differently. Anyone reading the last part of the log would struggle to figure out which was the you and which was the bot.
However, I am sorry that you thought my suggestions I left in the chats I had with Demonica for amending your bot to have a chance in the Loebner Prize was some kind of attack. It was genuinely intended as constructive feedback for you to improve your chances in the contest.
(https://assets.biztimes.com/uploads/2019/06/Einstein-insanity.png)
-
@WriterOfMinds - Thanks for your constructive feedback, this is really useful for me. One of the reasons I enter contests like this is to see how Kuki can be improved.
We talked about pets; she told me her budgie used to scare her. A budgie doesn't seem very fear-inspiring to me, so I asked followup questions, and she couldn't really tell me why he was frightening.
The pet budgie is part of the "human" version of Kuki rather than the publicly availble one on the website and so I have neglected this part of her. I can add follow up statements about how she was afraid the bird might fly and crash into her.
Later in the conversation, she claimed to have been born in both West Philadelphia and Leeds, UK.
I haven't seen any of the logs yet but this usually happens if people talk about being "born and raised", as a jokey reference to the TV show, The Fresh Prince of Bel Air. Hopefully, the logs will be made available and I can check and possibly get Kuki to clarify that she is making a joke.
She claimed to have recently read the Bible, and when I followed up by asking "What was your favorite story in it?" she was able to answer, which was impressive.
Thank you. An easy way to out a bot is to talk about a subject like The Bible and then ask a follow up question about "it" to see if it can keep context. A decent chatbot should realise that in this case, "it" refers to The Bible. Most of the time, when Kuki receives input containing the word "it", she substitutes the word for the contents of her "it" parameter and asks herself the same question. So "What was your favorite story in it?" becomes "What was your favorite story in The Bible?" which she can handle. It's great to see this happen and I'm glad it worked well for you.
-
Hi all,
Thanks to all competitors. Here is the results :
1 - Frizella: Nb round(s): 9 average time: 403.055 s
2 - Demonica: Nb round(s): 9 average time: 309.7358888888889 s
3 - Kuki: Nb round(s): 9 average time: 700.7252222222222 s disqualified (not connected 5378 s)
Sorry Steve, Kuki was been disconnected a too long time. It is the rule. dura lex sed lex. However, if you had not this technical problem, your bot would have broadly win.
All results and transcripts are here: http://vixia.fr/turing_test/results.php (http://vixia.fr/turing_test/results.php)
Globally the challenge ran as expected, I was afraid of a big crash or bugs. There is only some minor defaults that I have to correct.
FEEDBACK ON THE COMPETITION:
The interface worked well, and none of the bots were unavailable or froze up during conversation.
I think the rounds could be 15 minutes long, instead of 25. I had made judgments about all the bots in under 6 minutes. The shorter time commitment might help encourage more participation.
Running the contest for 24 hours makes it easier for people from all time zones to participate, but it also spreads out the small number of human participants such that it is harder to make human/human chat matches. If most participants were from the US and Europe, you might consider contracting the time window to one that overlaps with daytime in those places. When the competition becomes more popular, you could expand it again.
Thanks for your suggestion. I will take them into consideration.
-
Sorry Steve, Kuki was been disconnected a too long time. It is the rule. dura lex sed lex. However, if you had not this technical problem, your bot would have broadly win.
That's absolutely fine. Kuki had a few problems with the rounds starting and stopping every 30 minutes and I had to restart the bot a few times during the day. In the Loebner Prize, I would start and stop the bot manually between rounds but didn't fancy doing that for a 24 hour competition! ;D
This was a fantastic contest and one I really enjoyed being part of. Congratulations to you for your incredibly hard work. O0
My only change would be that once you guessed either human or bot, that decision was final and you could no longer keep talking to it and change your mind.
It was great fun to be part of and an interesting warm up to the Loebner Prize (if it happens this year).
-
May I also use this opportunity to mention a few tips I've found over the years when taking part in the Loebner Prize, as I know that entering the contest is important to at least one person here ;). I mentioned a few of these in individual chats with the bots I had but will collate them here and add a few extra.
1 - Change the bot's name. Many judges will ask, "What is your name?". If the bot responds with the actual bot name, this will out it. For example, in these contests, I change Kuki's name to Millie or sometimes generate a random name each time (Susan, Janet, Paula) etc
2 - Try to add a delay to your bot responses wherever possible. If I say "hello" and get a response of "Hi there. How are you?" within a fraction of a second of pressing enter, I will instantly know it's a bot. Kuki adds a few milliseconds for each character in her response, to give the illusion she is typing her answer, so the longer the answer, the longer it will take Kuki to display it.
3 - Consitency. For example, if your bot claims to have a job, you should then be able to ask questions about its job. If the bot says it has a job and then says it is unemployed, that will be unbelievable and unconvincing.
4 - Knowledge of what it says. If the bot says it likes a certain TV show and the human asks about it, the bot should be able to answer basic questions about it. If it says, it likes Justin Timberlake, it should at the very least know who Justin Timberlake is.
5 - Impatience. If the human doesn't type anything, consider adding responses like "Are you still there?", "Why aren't you talking" or "Have you fallen out with me?" that are automatically sent every so often. I send one after 3 minutes of inactivity. This gives the illusion that the bot is proactive rather than waiting for constant input and worked for me in this contest.
6 - Make it humanlike. The main part of the Loebner Prize is to try and create a humanlike bot. One that is indistinguishable from a human. If your bot says it has 40 legs or is 5 metres tall and lives on the moon, it will perfom badly and finish in the lower part of the results. Personally, I've never been a fan of the bot pretending to be a human, as it's deceptive but that's what the judges are looking for and so that's what we have to do. I have 2 versions of Kuki, the publicly available one doesn't pretend to be a human and knows it's a robot, has a bionic eye, travels around the world on the internet and possesses an infallible memory but I have a second version that I use in contests which will say she is an 18 year old, university student from Leeds, UK who studies psychology, has a pet budgie, a friend called Michelle and like to go to night clubs etc.
7 - Make it "stoopid". If your bot is asked what pi is and gives 100 decimal places rather than saying something like, "Ooh I'm not sure. 3 and a bit?", it will again be instantly outed as a bot. This is another reason I'm not a fan of bots pretending to be human. I would much rather they provided useful responses. If I type "What is the population of Norway" into Google and it responds, "No idea. A few million I guess?", that answer may be humanlike but is neither useful or intelligent.
8 - Typos - Misspelling words gives a humanlike appearance. Just a few simple typos like "What is teh time?" will suffice. Don't go overboard with them or I've found this has the opposite effect and makes your bot look like it's broken.
9 - Knowledge of current affairs/news - A popular tactic by judges is to ask about a big news story which any human would know. If your bot has never heard of covid (GPT-3!) then it's going to fail at being humanlike. Popular musicians, TV shows, celebrity gossip, current mnovies are also worth including.
There's probably a dozen more I've found over the years but I hope these are useful and I look forward to seeing you in the next Loebner Prize! 8)
-
That must have been you that had popcorn last night.
I've been browsing the logs, and fear you've confused me with someone else.
-
the thing is the kuki chatbot would have
failed the turing test easily.
here is a test i did.
somethings i would like to keep between
me and caroline
what? :(
-
the thing is the kuki chatbot would have
failed the turing test easily.
here is a test i did.
somethings i would like to keep between
me and caroline
what? :(
I think you posted this on another thread too. "Somethings" is not a word. Assuming you meant "Some things", Kuki would have responded, "Which things in particular?
GIGO ;)
-
yes i did. this was before that post.