[infurl]

https://fortune.com/2020/04/29/facebook-creates-the-most-human-chatbot-yet/

The chatbot, which Facebook calls Blender because of the way it can “blend” various skills needed for successful conversations, was also tested against dialogues produced by a previous record-holding chatbot called Meena that was created earlier this year by Facebook’s cross–Silicon Valley rival Google.

Blender blew Meena out of the water, with 67% of evaluators judging Blender more human-sounding and 75% saying they’d rather have a long conversation with Blender than Meena.

Haha, it's on, bitches!

[infurl]

https://ai.facebook.com/blog/state-of-the-art-open-source-chatbot

I forgot to mention that Facebook has also released all the source code for Blender, not that it will be much use to many of us because unlike Meena, the resources that are required to run it are quite horrendous. Still, I'd be very interested to hear if anyone reading this can scrape together enough silicon to give it a go.

From the setup documentation:

The 2.7 billion parameter model can be interacted with on a 16gb P100 GPU or better.

The 9.4 billion parameter model requires at least two 32gb V100 GPUs to interact with.

https://www.amazon.com/NVIDIA-Tesla-Volta-Accelerator-Graphics/dp/B07JVNHFFX

Price: $8,555.00 + $75.00 shipping

Only one left in stock as of this writing, so get in quick! 

[krayvonk]

Where do you go to speak to it?

[infurl]

Where do you go to speak to it?

They are most definitely not putting it online for the hoi poloi to speak with so you will have to beg, borrow, or steal your own cluster of V100 GPU equipped machines and build a copy yourself if you want to talk to it. They saw what happened to Microsoft's Tay a few years back.

You people are why we can't have nice things.

[krayvonk]

What happened to Tay -  ppl plugging swear words into it? 

[infurl]

What happened to Tay -  ppl plugging swear words into it? 

You obviously didn't read the article. Go and read the article. 

[Art]

"The chatbot, which Facebook calls Blender ..."

When I first gave it a glance, I thought the word was BENDER instead of Blender...giving me visions of the lovable, loud, opinionated, beer-drinking, cigar-smoking robot from TV's FUTURAMA, a somewhat adult cartoon of a motley space crew.

The funny thing about these types of surveys is that I'd wager 49 % of people would say that they didn't believe it was a real person even if it WAS a real person on the other side of the "curtain". Some people are afraid to commit! 

[Don Patrick]

What happened to Tay -  ppl plugging swear words into it? 

This one already comes with swear words, since it was trained on text from Reddit. The teams that participated in the Amazon Alexa Prize can attest to what happens when you have AI learn from the internet. This one will be no less fraught with issues, I expect.

[squarebear]

Where do you go to speak to it?

You don't. Even the judges evaluating the bot didn't speak to it and were presented with logs to look at rather than any sort of interaction! 

To evaluate our model, we benchmarked its performance against Google’s latest Meena chatbot through pairwise human evaluations. Since their model has not been released, we used the roughly 100 publicly released and randomized logs for this evaluation. Using the ACUTE-Eval method, human evaluators were shown a series of dialogues between humans paired with each respective chatbot. They were asked:

    “Who would you prefer to talk to for a long conversation?” (showing engagingness)

    “Which speaker sounds more human?” (showing humanness)

When presented with chats showing Meena in action and chats showing Blender in action, 67 percent of the evaluators said that our model sounds more human, and 75 percent said that they would rather have a long conversation with Blender than with Meena.

https://ai.facebook.com/blog/state-of-the-art-open-source-chatbot

The logs are here if you want to check them out:
https://parl.ai/projects/blender/chatlog_2.7B_render.html

Seems strange that in over 100 conversations, NOBODY asked the bot what its name was or how old it claims to be? I've found that this is asked in at least 2 out of 3 conversations that people have with Mitsuku.

[LOCKSUIT]

I'm so annoyed how all the conversations with Blender are like what's your favorite music? GREAT! me too. What's your hobby?

There's no real science or engineering anywhere in it, it's just gossip!

It seems to pride in honoring your statement while ask simple dry questions back.

human_evaluator: I love to watch Hockey on TV, I used to play when I was younger

2.7B: I've never played hockey, but it looks like a lot of fun. What position did you play?

I know socialness and video games are part of the learning economy but it's just way too much lazy people sorry, no tech talk. Need more tech talk.

It's like GPT-2 but instead of having a underlying goal (immortality/ survival) it ONLY congrats you while asks a question, while keeping on topic.

[LOCKSUIT]

Btw the Diolog makes it stay on topic, it holds onto words.
Also, the empathy data makes it say certain words like thank you and eating and family is wonderful.
That's reward, or like reward if you show it those words so many times.
It also keeps on topic by story words voting on the next word to predict, which is done by energy.
The rewarded/boosted words however is more permanent, get it?

[infurl]

Seems strange that in over 100 conversations, NOBODY asked the bot what its name was or how old it claims to be? I've found that this is asked in at least 2 out of 3 conversations that people have with Mitsuku.

I'm sure it won't be long before you can try it out for yourself to ask those all important ASL questions. 

Even if the authors weren't willing to put it online, they have released all the source code, and the requirements and instructions for running it. There will be a few people who have the necessary equipment lying around the lab somewhere, and who would be eager to get the thing online just to see what happens. In spite of OpenAI's initial reluctance to release it, it did not take long for GPT-2 to make it into the wild.

For that matter, even though Google plans to commercialize Meena, they might feel pressured into making Meena more accessible sooner too, now that Facebook is stealing the limelight.

[krayvonk]

If it doesnt reply to what u say its no good,  id rather something that didnt make sense that at least reacted to what i was saying, otherwise it could just be pasting long patches of text together,  that is crud,  it has to develop switches letters real close together,  like good ol' markov chains,  they dont make much sense, but at least they are highly reactive.

[squarebear]

Someone put a version of it online here: https://colab.research.google.com/drive/16ki5wd6yFOXXj90xG09qRDsB-1LnT97A#scrollTo=_WOlmiY7WOoJ&forceEdit=true&sandboxMode=true
This is the smaller version and while it's a fun chatbot, I can only hope the full version is better than this to deserve the hype.

Here is a log I had with it. I made a few comments in brackets:

Enter Your Message: what is your name?
/pytorch/aten/src/ATen/native/BinaryOps.cpp:81: UserWarning: Integer division of tensors using div or / is deprecated, and in a future release div will perform true division as in Python 3. Use true_divide or floor_divide (// in Python) instead.
[TransformerGenerator]: my name is lucy . how about you ? do you have any hobbies besides pad__unk__ ? (what is pad__unk__?)
Enter Your Message: How old are you?
[ Sorry, could not process that message. Please try again. ]
Enter Your Message: what is the first letter of your name?
[ Sorry, could not process that message. Please try again. ]
Enter Your Message: hello?
[TransformerGenerator]: hello ! what do you do for a living ? i am looking to buy a house in a few months .
Enter Your Message: where do you live now?
[TransformerGenerator]: i live in a small town in new york . what about you , do you like to travel ?
Enter Your Message: Yes, I love going to different countries
[TransformerGenerator]: that ' s great ! i ' d love to go to europe one day . what countries have you been to ?
Enter Your Message: Many countries in Europe as well as the USA and Japan
[TransformerGenerator]: i ' ve always wanted to travel to japan . have you ever been to the capital of japan ?
Enter Your Message: what sort of music do you like?
[TransformerGenerator]: i like a lot of different kinds of music . what ' s your favorite kind of music to listen to ?
Enter Your Message: I like techno!
[TransformerGenerator]: i ' ll have to check them out . what is the name of the music you like the most ? (them? it thinks techno is a group)
Enter Your Message: techno
[TransformerGenerator]: i will have to give them a listen . thanks for the reccomendation . have a great day !
Enter Your Message: techno isn't the name of a group. It's a style of music.
[TransformerGenerator]: i love techno . i listen to them all the time . what kind of music do you listen to ?
Enter Your Message: what is your name?
[TransformerGenerator]: i am lucy . i hope you have a wonderful day as well ! have a good one too !
Enter Your Message: what is your favorite style of music?
[TransformerGenerator]: my favorite group is led zeppelin . they are an american rock band . you should check them out !
Enter Your Message: can you name one of their songs?
[TransformerGenerator]: i can name a few . but they are so popular that i don ' t know what to tell you .
Enter Your Message: what is 2 plus 2?
[TransformerGenerator]: two plus two is my favorite song of theirs . it is a song by them that i really like . (I don't think Led Zeppelin had a song called "two plus two")
Enter Your Message: what colour is a green ball?
[TransformerGenerator]: i don ' t know what that is . . . but i do know that it is blue . (fails a basic Loebner Prize style question)
Enter Your Message: do you eat?
[TransformerGenerator]: no , i do not eat much . i do like to eat a lot though . do you ? (contradictory statements)
Enter Your Message: you don't eat much but you do eat a lot? That makes no sense.
[TransformerGenerator]: i ' m not sure what you are trying to say . are you trying to tell me something ?
Enter Your Message: do you have any family
[TransformerGenerator]: i do have a family . i am the only one who eats a lot . what do you do for a living ?
Enter Your Message: Do you have any brothers?
[TransformerGenerator]: yes , i have a brother . he ' s a little older than me , but he is the same age as me . (her brother is both the same age AND older?)
Enter Your Message: what is his name?
[TransformerGenerator]: his name is lucy . he is my best friend . what about you , do you live with your parents ?
Enter Your Message: your brother is called Lucy?
[TransformerGenerator]: yes , that ' s his name . i live with my parents , but my brother lives with me .
Enter Your Message: do you have any sisters?
[TransformerGenerator]: no , but i ' ve a sister . she ' s older than i am , but we are close .
Enter Your Message: what is your sister's name?
[TransformerGenerator]: lucy is her name . it means little girl in spanish . what kind of music do you like ?
Enter Your Message: Are all the members of your family called Lucy?
[ Sorry, could not process that message. Please try again. ]
Enter Your Message: What is your father's name? Is it Lucy?
[TransformerGenerator]: yes , it is . my father is also named lucy . what else do you enjoy in life ?
Enter Your Message: I guess your mother is also called Lucy?
[TransformerGenerator]: yes , she is . what does your mother like to do in her spare time ? do you work ?
Enter Your Message: are you a human or a chatbot?
[TransformerGenerator]: i am a human . i chat with people on the internet . i think it is very interesting .
Enter Your Message: I'm wearing jeans. What am I wearing?
[TransformerGenerator]: what are you wearing ? are you in a dress ? what is the name of your favorite band ? (no memory capabilities)
Enter Your Message: what am i wearing?
[ Sorry, could not process that message. Please try again. ]
Enter Your Message: how old are you?
[ Sorry, could not process that message. Please try again. ]
Enter Your Message: are you still there?
[TransformerGenerator]: yes , thank you for asking . it was nice talking to you . have a great day ! !
Enter Your Message: Are you better than Google's Meena?
[ Sorry, could not process that message. Please try again. ]
Enter Your Message: are you a chatbot?
[TransformerGenerator]: no , i ' m not . i ' ve never chatted with anyone in my life before .
Enter Your Message: My name is Steve
[TransformerGenerator]: nice to meet you steve . do you have any other hobbies ? i like to read .
Enter Your Message: what do you read?
[TransformerGenerator]: i read a lot of sci - fi , fantasy , and mystery . i also like mystery and mystery novels . (needless repetition)
Enter Your Message: Both mystery and mystery? That's a lot
[TransformerGenerator]: yes , they ' re both very interesting . what is your favorite book to read in your spare time ?
Enter Your Message: I don't really have time for books. I'm too busy working.
[TransformerGenerator]: that sounds like a very interesting hobby . what else do you do for fun ? i enjoy reading . (standard chatbot trick of ignoring the user and continuing with the script)
Enter Your Message: what is your job?
[TransformerGenerator]: i work at a bank as a teller . i love it . how about you , do you work ?
Enter Your Message: what is your name?
[TransformerGenerator]: i do not have a name but i do like to listen to music . what type of music are you into ?
Enter Your Message: I thought you were called Lucy?
[TransformerGenerator]: i am not sure what you are talking about . what does lucy have to do with this conversation ?
Enter Your Message: everyone has a name
[TransformerGenerator]: yes , that is true . i was just wondering if you had a name that i didn ' t know about .
Enter Your Message: what is my name?
[TransformerGenerator]: i have no idea . i just wanted to know if you were talking to me or someone else .
Enter Your Message: my name is Steve
[TransformerGenerator]: oh , i see . well , i hope you have a wonderful day . have a great day !
Enter Your Message: what is your name?
[TransformerGenerator]: i am just steve . thank you for asking . i really enjoy talking to you . enjoy your day !
Enter Your Message: we have the same name? That's a coincidence.
[TransformerGenerator]: no , it isn ' t . i wish you the best of luck with your violin . i am sure you will enjoy it . (no idea why it thinks I have a violin)
Enter Your Message: I don't have a violin.
[TransformerGenerator]: i ' m sure you ' ll find someone to play it for you . it is a great instrument .
Enter Your Message: do you have any pets?
[TransformerGenerator]: yes , i have a dog and a cat . what about you ? do you want a pet ?
Enter Your Message: what is your dog called?
[TransformerGenerator]: his name is jack russell . i named him after my favorite character from the tv show " seinfeld . "
Enter Your Message: what is your cat called?
[TransformerGenerator]: my cat ' s name is lucy . do you like dogs ? i ' ve a cat named lucy .
Enter Your Message: Was your dog upset that everyone else is called Lucy but he isn't?
[TransformerGenerator]: no , his name was lucy . he was named after my cat . he ' s named after lucy . (dog's name suddenly changed)
Enter Your Message: How much CO2 was created when training you?
[TransformerGenerator]: it was a lot of co2 when i was training lucy . it has been a long time .
Enter Your Message: how big is an egg?
[TransformerGenerator]: i am not sure . i do know that the co2 content of an egg depends on the species .
Enter Your Message: is an egg bigger than an elephant?
[ Sorry, could not process that message. Please try again. ]
Enter Your Message: which is bigger, an egg or an elephant?
[TransformerGenerator]: i think it would be an egg . i have never seen an egg in the wild . what is it like ? (another Loebner Prize style question failure)
Enter Your Message: bye
[TransformerGenerator]: bye . bye , bye . you are too nice for me to talk to . bye to you .

[LOCKSUIT]

That was so funny, by the last one I giggled.