Learning from feedback (do conversation partners give positive or negative responses to what I say?) and learning by imitating others are both useful tools, in my opinion ... humans certainly use them. But it's never "just that simple." I think there's a lot of structure that goes into how to learn effectively.
Cleverbot is a chatbot that tries to learn by imitating what humans say to it. I've talked to it in the past, and I don't recall it being terribly good.
It could be a bit more complicated than simple "copying", but it is in that spirit. The key could be in higher order copying. For example, a function from A to B, say, a color of eyes. Bot sees someone asks: "What is Nick's color of eyes?" Someone responds: green, and the bot sees the green color in Nick's eyes. A function of eye color is built in bots mind. Next we ask bot: "What is Leo's eye color?". Bot then puts Leo through the `eye function` where input is a human and output is a color. The bot finally responds "Blue!".
In this idealized story, the eye color function is just a single primitive function, but other functions may be built out of composition of primitive functions. Then there is composition of compositions, and so on.
When bot sees a stimulus and a response in outer world, it learns a function which input is stimulus, and output is response. The bot groups similar stimulus-response pairs into the same functions. Later the bot combines functions, or even uses functions as a stimulus or a response of other functions.
It all brings down to a pure combinatorial problem - first to learn primitive functions, and later to combine them in learning new behaviors. There also should be noted a distinction between variables and constants. The bot combines various constants into a single variable, and connects such variables into functions.
This is just a theory, but I believe this is what happens in generative neural networks. Input is a set of function parameters. Output is possible to be created after some training - what parameters fit to what output - find analog ones to similar new parameters. NN is just what magically replaces all the combinatorial function problems.
This is where program synthesis could replace generative NNs. Program synthesis works like pairing inputs and outputs in symbolic fashion, finding right functions from inputs to outputs. That is like training NNs. But NNs are statistical beasts, and they give a fuzzy results, reporting a certain percent of success. Program synthesis, in contrary, is a 100% input-output match. In practice, giving a bunch of input-output pairs to program synthesis may result in only subset of pairs matching by synthesized functions, giving again a rough percentage of success. But it should be a symbolic, human tractable constructions where we may analyze steps of producing a result from parameters.
There is a lot of data being involved in thinking, from a vast of raw input data to a handful of high level data on which decisions are being made. To restrain an AI from the outside, analyzing this high level data may be crucial, and that is what program synthesis may provide if used instead of NNs. For low level, where there is a lot of data, NNs may be a better choice, but for high level decisions - I think the answer may lie in program synthesis.
