[white]

The second is a 'friend' test.

Me: My friend Eliza finally came over.
Me: Was she late?
Bot: The answer to the question (was she late?) is not known.
Me: Where is Eliza?
Bot: The answer to the question (where is eliza?) is not known.

I would argue that the above answers are not incorrect. The fact that Eliza finally came over does not need to mean she was late. Perhaps you have been asking her for a date once a day for a month, and she finally gave in and came over ...being perfectly on time.

The statement is written with a past participle and the question "Where is Eliza" is not. Eliza can be anywhere at the moment. But even if the question would have been rephrased to "Where was Eliza when she came over?", what would a human answer? Assumingly at your home but it isn't really specified. You could have been at a party at your friend's house when Eliza finally decided to show up.

[Genesis]

I would argue that the above answers are not incorrect. The fact that Eliza finally came over does not need to mean she was late.
Perhaps you have been asking her for a date once a day for a month, and she finally gave in and came over ...being perfectly on time.

Actually they are correct and you are proving the reason why. The test is designed to be ambiguous as possible, which would require the ai to make predictions or assumptions on what's happening.
Its the same way we humans do it. Just like you just said "finally" does not mean she was late.
If an AI can't make predictions with as little info as possible it can't be intelligent.

An intelligent AI would recognize that I was expecting her to come over from the statement "finally" and since it couldn't conclude whether or not she was late on "coming over" from the statement "finally" it would try to ask a question to put the pieces of the puzzle together just like we would. It would ask, "what time were you expecting her to come over?"

I have yet to see an AI chat-bot that can accomplish that task, if you can build an AI that can handle stuff like that. Then it would truly be the most intelligent AI in the world right now.

The statement is written with a past participle and the question "Where is Eliza" is not. Eliza can be anywhere at the moment. But even if the question would have been rephrased to "Where was Eliza when she came over?", what would a human answer? Assumingly at your home but it isn't really specified. You could have been at a party at your friend's house when Eliza finally decided to show up.

It doesn't matter how the question is phrased, we humans easily understand it because we understand 'context' and we make predictions based on it. If I were to say to you in some random IRC chatroom that A friend of mine finally came over and I asked you where they were. Your first thought would be 'with you' and 'at your house'

It doesn't matter whether you are right or wrong about your predictions, it matters that you did predict and that you were able to make connections.

 For example, You made connection with "coming over" and knew that the word/action of coming over almost all the time has to do with some one going to someone elses house.

And like I said, this test is more towards those developing AGI (Artificial General Intelligence). Like I also said, your predictions don't have to be right.
For example the time test. We predict the time all the time. We predict how long has passed since a point in time. We do this based on the sequence of events that have occured after that point in time.

We can approximately predict whether 15mins, 30 mins, or an hour has passed. Most of the time we are accurate, but we are not 100% right. If you give an AI specificity then it would be clueless when it encounters situations that lacks it. But if your AI can learn through situations with the least bit of specificity and information, only then can it be true-ly intelligent.

By the way, that isn't fully the 'friend' test either.

[Genesis]

Right now, here are what all chat bots can do.

Me: I am at my friend's house.
Me: Where am I?
Bot: You are at your friend's house.

Pretty primitive, no reasoning involved, just a database.

[sunama]

Right, I am now spending a little time on dealing with "context of current conversation".

I'm looking at the the input/output suggested by Genesis.

The time test: a good language processor should be able to answer this question, so I shall definitely be working on this, over the next month.
The friend test: now this involves 2 concepts: 1. ambiguity handling as well as 2. context of current conversation.

Ambiguity handling is the most difficult.
IMO, for an AI to successfully deal with ambiguity, it will take a few years of work and should not be dealt with in the early years. For this reason, I won't be dealing with ambiguity just yet.
Computers tend to not handle ambiguity too well and when they are subjected to ambiguous input, they tend to come up with garbage answers.

I would also say that if you asked most human beings (about the friend test), they would also not know those answers.

[victorshulist]

Right now, here are what all chat bots can do.

Me: I am at my friend's house.
Me: Where am I?
Bot: You are at your friend's house.

Pretty primitive, no reasoning involved, just a database.

Well, that's an interestng remark.   You do realize that there is almost an infinite number of  'levels of difficultly' from this example and up right?

It depends on how flexible the system is.

*IF* the above is done with a simple template system like such....

input: (x) is at (y)
input : where is (x)

response =>  (y)

then of course the sysem will fail horribly.

In fact, we talk about a "Turing Test"... it is joke of course with current technology.

I would bet anyone that I could detect a bot within 60 seconds, even if it would be, what i call a "Telephone Number Turing Test"
that, even limiting the discussion to just stating people's phone numbers and asking for them back...60 seconds I bet I could break any bot.

can anybot do this even.....

John's phone number is 111-2222.
What's John's phone number? ------------------ easy, probably many bots can do with simple "template system"   (x)'s (y) is (z), what is (x)'s (y), answer: (z)
What's John's number?
What's John's # ?
John's # -- what is it ?
do you know John's number?
u know john's number?
what's john's telephone number?
Do you know john's *cell*  number?

...and on and on the list goes.

I have to laugh when people are talking about deep inference when powerful language processing is not even complete yet.

Develop an extremely powerful system that can pass a 60 second Telephone Turing Test  (with someone that can REALLY ATTACK the system.. that knows the weaknesses of chatbots), and it if can pass even 60 seconds.. then you have made real progress in naturual language understanding.

THEN we can start talking about inference.  That is, knowing what is meant by "finally came over", etc.

I  DO like a lot of your thinking though.. especially the comment regading the fact that it doesn't have to make the correct predictions.. just make some connections... those connections that people would make.  But I really think NLU has to be completed first.

[sunama]

The problem with dealing with ambiguity and effectively guessing or assuming what the user actually means, is that you end up with a system which could potentially end up inferring something that has absolutely no relation to the original query.

I'm not sure if ambiguity will ever be handled correctly by a computer, for the above reason...but only time will tell.

[victorshulist]

Oh they can and will deal with unbeleivable amounts of ambiguity.   

[DaveMorton]

“That’s right!” shouted Vroomfondel, “we demand rigidly defined areas of doubt and uncertainty!”

Hiya, Victor! glad to see you!

[victorshulist]

I love the quote Dave ! "“we demand rigidly defined areas of doubt and uncertainty" -- going to use it as an email footer 

Good to see you to sir!