[DemonRaven]

I have been studying English and found out that there are two camps, prescriptivists who believe that language has rules that don't change and descriptivists who believe rules do change. In Chat-bots this is especially important because we use a lot of English and other languages to program/train them. So which camp do we use for chat-bots? Hard and fast rules that don't bend or rules that do bend and evolve. I am of the opinion that we must allow for language to evolve and that includes the rules for English. What do others think? Do you think we  need  hard and fast unbending rules or rules that do change when using natural language processing? I am curious and thought it might be a interesting discussion.

[AndyGoode]

I have been studying English and found out that there are two camps, prescriptivists who believe that language has rules that don't change and descriptivists who believe rules do change. In Chat-bots this is especially important because we use a lot of English and other languages to program/train them. So which camp do we use for chat-bots?

What type of rules are you talking about? All languages change slowly, like how English speakers today would not understand Shakespearian English from 400 years ago, and in modern times many slang and technical words are always being added, but grammar almost doesn't change at all, as far as I know.

By far I believe your biggest problems with chatbot programming are going to be machine understanding and commonsense reasoning, not grammar or even vocabulary.

Are you a native English speaker?

[MikeB]

I think you need to account for as many combinations as possible. For Eg. "I baked a cake" could be said "Cake, I baked".

The words themselves could be in different languages, but there are a couple of different ways of saying the same thing. There's not infinite ways so that's a bonus.

[HS]

I think language should be the proverbial reed that bends. Certain rules should act as foundations. Then there should be a spectrum of rules which increase in flexibility. Some great things can be done with language because of its unchanging rules, but in other instances rules can be tied in knots to great effect. Chatbots need both to get the most out of language. Like this: (one of my favorite videos!)

[DemonRaven]

I have been studying English and found out that there are two camps, prescriptivists who believe that language has rules that don't change and descriptivists who believe rules do change. In Chat-bots this is especially important because we use a lot of English and other languages to program/train them. So which camp do we use for chat-bots?

What type of rules are you talking about? All languages change slowly, like how English speakers today would not understand Shakespearian English from 400 years ago, and in modern times many slang and technical words are always being added, but grammar almost doesn't change at all, as far as I know.

By far I believe your biggest problems with chatbot programming are going to be machine understanding and commonsense reasoning, not grammar or even vocabulary.

Are you a native English speaker?

The rules are grammar rules for English and how words are used. Yes I am a native speaker but I felt I needed to brush up on my old high school studies to keep my mind fresh. I am a old fart and it has been decades since I studied English. I know that at least in one case the personality forge Grammer rules are used to a point. But yes language does evolve you are correct.

[Don Patrick]

I have spent a lot of time programming grammar. There are certainly rules to be found that follow certain logics and consistency, but there also exist exceptions to each and every one of them. Grammar is really just a form, a guideline of courtesy. Contrary to what I was taught in school, English does not strictly follow the subject-verb-object order. Sherlock Holmes would make verb-subject-object statements "Have you your gun?", and even relative clauses like "the program (that) I made" are  object-subject-verb order, never mind even Yoda. I found out fairly early on that grammar rules only get you so far (let's say 60% of the way to understanding). It's not bad but not sufficient. After that it's semantics, which also feature rules, as well as exceptions like expressions and idioms whose origins are long forgotten. Even if we just look at spelling rules or pronunciation (which for some senseless reason is not written pronOUnciation), there is no consistency because English is a mix of Latin, Greek, Old English, and French.

People used to program AI with context-free grammar, which could only process full and properly written sentences. I do not find this an unreasonable approach in principle, it does allow one to communicate intelligently with AI, it just requires more effort of humans and can't be applied to e.g. social media where the money is at. At the other extreme, people try to "train" AI on text with no hint of rules. This produces the nonsense poetry and ungrammatical strings of text that we've seen the last decade. It can guess a fair way but is only as reliable as a statistical average: Most of the time, it's right, but some of the time, it's completely bonkers.

I believe there are basic rules, I also believe that language changes. I do not believe that one can program every (new) exception to the rules, nor do I believe that a program can learn language without giving it any handles. I choose to program the main rules, and then use learning mechanisms to learn the exceptions to them. If and when language evolves beyond recognition, computers won't be running Windows programs anymore anyway.

[AndyGoode]

Like this: (one of my favorite videos!)

Good video.

The only other comment I believe I could add to this thread that is likely to be valuable is that there exists a generalization of the 10 English sentence forms that will account for swapped word orders and more. I came up with that generalization on my own but I don't want to disclose it until I can write a paper on it. I don't mind letting ambitious people discover it on their own, though.