[LOCKSUIT]

The same discoveries you can make in Vision can be made in Text. For example, that cats are similar to dogs either because of their looks [zook]=[mooq] or their neighbor contexts's looks dog_[bowl]=cat_[bowl]. Another: That cats usually lick and rarely sleep. Another: The topic as of recency is all about cats, cats will probably come up again probably. Cats. "Useless", eh? Beat that.

The 144 attention heads, I never looked at them like that. Interesting. It seems they look at certain words. One for example saw James [Vig] is the [son] of John _ .....the attention head for Son/mom/etc saw Son was said, allowing/causing the last name attention head to look for the 2nd Name and predict 'that'. Or something like that.

[Korrelan]

The same discoveries you can make in Vision can be made in Text.

Ok... I disagree but that's fine... carry on.   

[LOCKSUIT]

What novel type of discovery can you discover in Vision, that you cannot in Text?

[ivan.moony]

Vision is 2D. Text is 1D. A sequence of finite 1Ds is 2D. Hence, 2D can be expressed in 1D. And so can 3D and so on. This probably causes a mess until NN finds out how to stitch multiple 1Ds into a single 2D, but once it does, it can do stuff in 2D over text dimension, right?

[LOCKSUIT]

I don't believe image is a sequence, I think objects in it are feature-ized and are able to 'not' save anything else but the board dice near the ex. upper right corner.

Well, it we feed it image data, and teach it how to look at it, then yes....

[infurl]

Reality doesn't care what you believe. As long as you keep making such wild guesses based on your own limited experience and knowledge, and assuming they are correct without actually verifying them, you will continue to compound your errors and wander even further from any useful results.

You should familiarize yourself with Jeff Hawkins' research. His company, Numenta, has pushed our understanding of how the brain really works further than anybody. What they have found is that all types of perception are active processes which combine prediction with feedback. Perception has at least four dimensions because it is always tied to both space and time.

[LOCKSUIT]

Oops, i didn't mean vision isn't a sequence, i meant image isn't. Obviously you don't store image object parts as lines of the image, you need to store segmented objects! Ex. you store a diamond object node/capsule:

not like this:
                      -------
--------------------------
--------------------------
---------

but like this:
        -
       ----
    --------
      -----
        -

Now this, this is a sequence, you can see one part of the diamond and generate the next, yes.....however to do it correctly you must look at it as 2D objects/capsule nodes once again, back to the thing I illustrated above. We don't store full image lines in nodes.....


I already knew the brain mixes all sensory types for improving its prediction. And how the brain uses feedback/data over time to specialize what kind of feedback/data it collects. That gives it "more data" technically. And when in-confident, will seek more data from some next best domain.

[silent one]

An image Is a sequence,  what if it was a photo of a newspaper.

[LOCKSUIT]

A newspaper is an image, you can only read it if you scan over it word by word, which is a video therefore. If you widden your attention to the whole page and try to read it all in 1 frame, you can't, all you'll see/read is "wall o text" XD.

And k, answer that question! xd

[silent one]

Markov chains work in graphics as well as text,  So  I agree with Locksuit,  but Korrellan is onto some other way of thinking that's just as good.

[LOCKSUIT]

Referring to the images in this pdf https://arxiv.org/pdf/1911.01547.pdf

Text/GPT-2 and Vision both have an "ARC" set of tasks. I call it "infinite tasks of things" or "Internet of Things" because you'll see it is absurdly flexible. See attached a list of tasks for text I made/collected. This is my "ARC" text set.

Whether vision or text, it works the same way, you either show it multiple examples or tell it "translate x to french" (assuming it has seen multiple examples) and it will do just that. It's a temporary priming/control or "way of thinking" for the net.

If we look at the start of my book I'm creating (see other attachment), we see Syntactics, Semantics, BackOff, BPE, etc can learn patterns, however we (not the algorithm) tell the algorithm to find these (priors), the algorithm should be able to discover these! Is there more it can discover? Semantics, BPE, what else is there? Can it find more!? Look at the text tasks I made, translation, word unscramble, find rare word in sentence, mom/dad/son same last names pattern, etc. There is more! But they are less common, so they seem like the last end of the hog's shoe and are hard/unnoticeable to find. In fact I think the main AGI net will be based on the most common priors, and will have to emulate other priors using the more common mechanisms. It will improve prediction accuracy less if less common, unless you can handle 1,000 types of pattern tasks.

GPT-2 has 144 attention heads it learns and then they look for "patterns".

As for rotating an L shaped objects in text, vision does just that, if you do just that in text then it is really vision after all, so if we want to do it in text the way text was designed for, it would be more like this: We have an L shaped object with the top of the L sticking up, please rotate it clockwise, what do we have now? Say answer: The top of the L shaped object is sticking out to the right.

I'm still checking if these "many tasks" are really just the few mechanisms listed in my attachment BOOK.txt - it could be that there is some pattern among them.

[LOCKSUIT]

And the "AGI net" seems to be only the top most common patterns. Why? Frequency/syntactics, semantics, BPE.....Does it not handle all other tasks? When you call/ask for the task "translate to french", this makes my architecture only use the recognized nodes, it only requires my architecture see!, and it uses energy to cause tasks temporarily, and highlights french too using activation energy temporarily, further after the task is said, you feed it a subject to manipulate, ex. dog, and it translates it to french "dog" version, because the 'dog' was activated and so was the french version. So it seems using just energy activation you can control not just the task but the subject fed in, using just a simple architecture?

There is many patterns in text/vision, many are less used, few are common, the brain architecture I'm guessing is only made of the most common patterns to naturally find those without being told to store/find them, How can it do this? It emulates, but how? It must be that the most common patterns make up the other bazillion patterns! A sequence of tasks. Now, even the net mechanisms have patterns, frequency, semantics, BPE, more data, all give u more data! And they blend the nodes as representations. Thee bazillion other tasks/patterns that 'reoccur' also give you more data and help prediction. Just saying. Anyway, hmm. I couldn't imagine a brain mechanism to make last names the same, 2nd names the same, or first, or etc etc etc, it must be more general than that.

Maybe you can help. How does AGI say the 3rd name as the same as relative? Ex. Jim Fan [Lee] has a son named Roy Tan [Lee]. The son and the other name clue it in. But how exactly. It could be said like Jim Fan um oh ya Lee. It knows names are odd, so it's looking for "names" and counting to 3? That would require a 123 node and a low frequency threshold circuit, and scanning memory maybe. ??

[LOCKSUIT]

Just realized, vision has one discovery edge over text, but only increases accuracy of prediction, not a fundamental difference do note!:

If you have a hierarchy of [visual features], pig is is made of teeth, hooves, tail, eyes, retina, pink, has a genital, etc. But in a text hierarchy, well, pig isn't made of those!!! It's made of pi+g, and pi is made of p+i. And no, you DON'T make a Concept Hierarchy, the brain has no such. So how does text know pig has legs? Oh it can, by ex. seeing it in text or generalizing semantically. Just not as often as vision.

This is why you can activate a node and all it's parts can be activated (legs, pink, teeth, etc) more than similar features (sheep, dog, monkey, goat).

[ivan.moony]

Vision is one abstraction. Hearing is another. So is smell, and so on. And all those abstractions relate in outer world.

Now, text is interesting. Text on its own isn't very special, it is a merely arranged system of letters. But in correlation to our sensing of outer world... Text is artificial abstraction used to represent vision, hearing, ... We draw similarities between words made of letters, and tangible concepts in vision, sound, ..., so we can communicate those concepts for doing serious business, or simply enjoying fun. Words become merely keys for tagging real concepts.

It sounds so banal when I read what I just wrote, but I'll publish it anyway.

[LOCKSUIT]

Text [alone] tells you what flowers smell like, vision [alone] doesn't :P

1up for text now!