A possible revelation?
See the image attached first, both nets do the same thing for this case. This is a very simplified "brain' storing observations which have/are patterns. It compresses data because it doesn't store things twice. It learns/ updates frequencies of strings like a, ab, abc, the, the cat, etc by strengthening connection channels. It can predict the next letter or word that follows any of these strings. It's predictions may be 't' has been seen to follow 73 times. 'p' 8 times. And 'z' 1 time.
But what about prompts like:
"Julie Kim Lee has a mom named Taylor Alexa [Lee]"
"A word similar to love is: [hate]"
"parrot me: [parrot me]"
"Please summarize 'the cat was playing all night with its cat friends': [cats partying]"
"How many times does 'a' appear in this question?: [4]"
"Predict the next words: If the dog falls off the table onto the [floor, he may not be alive anymore]"
"if cats are huge AND cute then say 'hi' to me: []"
"Find me the most [rare] word in this sentence"
"[refine the first 4] words of this sentence please"
"super is for superman and bat is for [batman]"
"write me a book about cats that is 400 words long: []"
"Dan likes Jen and I believe Jen likes [Dan]"
"Cats are dogs. Hats but clothes. After god before. Look and ignore. Wind crane gust. jog cat [run]."
"The cat (who was seen in a dumpster last night) [is eating catnip]"
"Can cars fly? [No]."
"Dogs cats horses zebra fish birds [pigs]"
"remove the last [word]"
"scramble the last [orwd]"
"highlight the 2 most related words in the next sentence: 'the [cat] ate his shoes and the [dog] ran off'"
"[Segment [this sentence]] please"
"King is to man as Woman is to [Queen]"
"[please] inflate this [cool] sentence"
OOV WORD > "I love my F7BBK4, it cleans really well, so I told my friend he should buy a [F7BBK4]"
It seems that the network has to take examples of sentences as shown above and learn how to predict the next word. For the one "Julie Kim Lee has a mom named Taylor Alexa [Lee]" it would learn to make neural paths open for the 3rd word if sees the context, or something like that. These are like "rules" of where to pay attention.
