Ai Dreams Forum

Software & Hardware => General Hardware Talk => Topic started by: frankinstien on May 18, 2020, 01:33:38 am

Title: My HAl Rig
Post by: frankinstien on May 18, 2020, 01:33:38 am
Since I believe the neural network approach isn't the most efficient use of digital resources because it effectively is a linear search in a problem or knowledge domain, no matter how much pruning of a network you do. So I'm using a good deal of RAM, 128GB, and two 10 core Xeons. This doesn't mean I don't use GPUs, I do and its a Vega 56, I use it primarily to work with fuzzy logic and fast index calculations as well as process video and audio data. Below is a diagram of how I organize memory from RAM, SSD to hard drives.

(http://wraithbots.com/wp-content/uploads/2020/05/HalRig.png)

The SSD is a 1 TB drive and it stores data from a noSql database I engineered that uses O(1) lookup indexing. The Hybrid hard drive is more of a backup of the SSD but will store very large datasets like long video. You'll notice a block called "Cross Domain Harness" I really wanted to avoid a lot of serializing and deserializing between processes, so the harness allows the dynamic injection of various programs under a single domain. Since when I prototype I haven't necessarily integrated all the pieces yet and I don't want to deal with the latency of serialization. This way I can pass data by reference so long as the programs have imports to the datatypes used. It makes life a lot easier to troubleshoot particularly when the amount of data loading is in the 10s of GBs.

The machine was built three years ago so upgrading is starting to seem like a good idea since AMD's processors are reasonably priced, however, the RAM prices haven't dropped by much. So if I did upgrade it's a 50% improvement in performance using the Ryzen 9 3900X 12-core. Because my current machine is a dual-socket Xeon board it has 16 memory slots which allows me to use lower-cost 8GB sticks. In fact, I do debate if it's worth putting up the money for the newer Ryzen or just getting 16GB sticks to work with larger datasets?
Title: Re: My HAl Rig
Post by: infurl on May 18, 2020, 03:18:20 am
That's a pretty good setup that you've got there. I'd hold off on upgrading your hardware until you can at least double your performance. In the meantime, work on improving your algorithms or implementation because ultimately that's where the most gains are to be made. When it's too easy to keep getting faster hardware, it's easy to forget that.

What exactly can you do with it at the moment? You can store an awful lot of data, is there any particular problem that you are trying to solve?
Title: Re: My HAl Rig
Post by: frankinstien on May 18, 2020, 02:49:57 pm
What exactly can you do with it at the moment? You can store an awful lot of data, is there any particular problem that you are trying to solve?

The objective is to build the infrastructure for episodic memories and process and store information across a spectrum of sensory inputs, so it can be freely associated, yet be very consumable for digital processing.
Title: Re: My HAl Rig
Post by: Yervelcome on May 18, 2020, 04:22:34 pm
Do you have a write up where I can read about it?
Title: Re: My HAl Rig
Post by: frankinstien on May 18, 2020, 06:29:17 pm
Do you have a write up where I can read about it?

I don't have any formal material fully explaining the concepts of software right now but it's a "To Do" in the near future.
Title: Re: My HAl Rig
Post by: infurl on May 18, 2020, 10:15:31 pm
Do you have a write up where I can read about it?
I don't have any formal material fully explaining the concepts of software right now but it's a "To Do" in the near future.

Since you've piqued our interest I daresay the coming barrage of questions will prompt you to make a start very soon.  ;D

I'm curious about the custom solution that you're developing. You described it as noSQL and having O(1) access characteristics which implies hashing. You said you were using your GPU for indexing which I understand to mean that you are using it for generating your hash keys. I can guess why you would want to do that if you're interested in turning media like video into associative memory.

How about the symbolic side of things. Have you done anything with relational databases or triple stores such as Resource Description Framework (RDF)? What about ontologies?
Title: Re: My HAl Rig
Post by: frankinstien on May 19, 2020, 02:15:35 am
I'm curious about the custom solution that you're developing. You described it as noSQL and having O(1) access characteristics which implies hashing. You said you were using your GPU for indexing which I understand to mean that you are using it for generating your hash keys. I can guess why you would want to do that if you're interested in turning media like video into associative memory.

Yes hashing is used but it's a bit more exotic than other applications since it has to work with fuzzy sets of data. For the video, the approach is breaking visual data into manageable pieces that realize into generalizations where those generalizations allow for associations.

How about the symbolic side of things. Have you done anything with relational databases or triple stores such as Resource Description Framework (RDF)? What about ontologies?
The ontological framework is custom as well and uses NoSql for long term storage and is structurally a graph database. The entire approach is object-oriented and allows for sperate threads to crawl through knowledge domains as well as allowing for updates to propagate through the graph instantly. It was inspired by Roget's Thesaurus. It's used to evaluate speech or text where that data can be analyzed as to what kind of bias generalizations can be derived from the hierarchal nodes. Words are endowed with properties and allow for validation of a word's use within a sentence. This helps in quantifying logic, context, and meaning of a sentence or groups of sentences.
Title: Re: My HAl Rig
Post by: infurl on May 19, 2020, 03:28:18 am
Yes hashing is used but it's a bit more exotic than other applications since it has to work with fuzzy sets of data. For the video, the approach is breaking visual data into manageable pieces that realize into generalizations where those generalizations allow for associations.

How does that differ from say fingerprinting a photo or music to find duplicates, or Google's ability to identify copyright material in YouTube videos?

The ontological framework is custom as well and uses NoSql for long term storage and is structurally a graph database. The entire approach is object-oriented and allows for separate threads to crawl through knowledge domains as well as allowing for updates to propagate through the graph instantly. It was inspired by Roget's Thesaurus. It's used to evaluate speech or text where that data can be analyzed as to what kind of bias generalizations can be derived from the hierarchical nodes. Words are endowed with properties and allow for validation of a word's use within a sentence. This helps in quantifying logic, context, and meaning of a sentence or groups of sentences.

It sounds like you are doing more than just sentiment analysis then. Do you have some examples of inputs and outputs to illustrate what you can use it for?
Title: Re: My HAl Rig
Post by: frankinstien on May 19, 2020, 06:43:54 pm
How does that differ from say fingerprinting a photo or music to find duplicates, or Google's ability to identify copyright material in YouTube videos?

I'm not sure what Google is doing to uniquely identify visual images to protect copyrights, but the method I'm using is based on fractal dimensions.

It sounds like you are doing more than just sentiment analysis then. Do you have some examples of inputs and outputs to illustrate what you can use it for?

I will be putting together some examples and videos for my website and I'll post them on this forum as well.
Title: Re: My HAl Rig
Post by: frankinstien on October 30, 2021, 07:32:00 pm
Well, I've waited and waited and the chip shortage continues.  :-\ So, I coughed up some cash and have upgraded my HAL rig to a Ryzen Threadripper 3960x 24 cores 48 threads 3.8ghz with 256 GB of ram at 2133 MHZ. Also added a 2TB SSD NVMe PCIe that can do R/W up to 3,400/3,000 MB/s. I'm sticking with my Vega 56 for now. So this rig is three times faster than the older one with the current GPU, change the GPU to a NVIDIA GeForce RTX 3090 24GB and I'm 6 times faster. :35:

Oh and this rig is actually illegal in California as a PC, but as a server it's legal. Welcome to my world!  :idiot2:
Title: Re: My HAl Rig
Post by: WriterOfMinds on October 30, 2021, 09:52:54 pm
I don't know that we even got through the crypto mining craze before this chip shortage started plaguing us  >:(  When I built my last PC, I went to the local parts store and whole sections of shelf in the GPU area were bare.

Quote
Oh and this rig is actually illegal in California as a PC, but as a server it's legal.

What's up with that? Are they trying to limit electricity consumption?
Title: Re: My HAl Rig
Post by: frankinstien on October 30, 2021, 10:31:57 pm
Quote
What's up with that? Are they trying to limit electricity consumption?

That's exactly why. Dell won't ship some Alienware rigs to California, Colorado, and other green states for the same reason. Gamming is going to suck in those states for a while.
Title: Re: My HAl Rig
Post by: frankinstien on November 02, 2021, 12:04:01 am
I added an additional NVMe Internal SSD - Gen4 PCIe, M.2 2280, 3D NAND to the rig. The biggest problem I face with storage is serialization and deserialization which is why I use soo much ram. When its just ram I can simply reference an object's memory location, so I can build very intricate indexes that reference the memory location and not the object itself. If you try and store those references in a drive it will make redundant copies of the object itself because the memory location of the object in ram will not be relevant if and when you reload the object.  So I replace the references with an ID of the object in a OO database. But Intel has Optane DDR4s (https://summerofhpc.prace-ri.eu/new-era-of-persisting-data/?utm_source=rss&utm_medium=rss&utm_campaign=new-era-of-persisting-data)! literally, solid-state storage in a Dimm form factor that plugs into a ram slot like SDRAM! The Optane Storage technology has speeds of 170 to 360 nanoseconds compared to DRAM which operates at around 90 nanoseconds, but it's still pretty fast compared to the fastest SSDs. With this kind of concept, I could persist the memory references as if they were in RAM and not have to map from object IDs to actual objects that are deserialized from disk storage. Yes, those Optane DDR4 persistent memory modules store data even when the machine is powered off!

Unfortunately, these gems will only work with the latest Intel machines where the core densities and speed are nowhere near AMDs CPUs at the same price.  :( 
I found a 256GB DDR4 Optane module on eBay (https://www.ebay.com/itm/124663799694?epid=12035260732&hash=item1d068a9f8e:g:tSwAAOSwWZdfkRkM) for $400!

Here's a video that explains it in a real implementation:

Optane (https://www.youtube.com/watch?v=uHAfTty9UWY)
Title: Re: My HAl Rig
Post by: frankinstien on November 02, 2021, 10:09:48 pm
Here's another video on the Optane DDR4 technology. It turns out that Micron has sold the fabrication facility that makes Optane to Texas instruments who will not be using it to make Optane type chips for the DDR4 form factor or any other app inclusive of SSDs, however, Intel no longer will sell Optane SSDs either. Intel will be relying on its Rio Rancho, New Mexico, facility for Optane chips for the DDR4 form factor. So Micron is out of the 3D XPoint business, but Intel and Micron still retain the intellectual property of 3D XPoint. Maybe Micro will find a buyer for the 3D XPoint IP or maybe CXL (https://www.computeexpresslink.org/) will deliver something even more amazing.  It would be nice if there could be some bio update for xTRS4 motherboards to use the DDR4 persistent data modules, the ability to read storage like ram and not have to serialize and deserialize objects but simply reference them saves tons of CPU cycles.

Optane DDR4 (https://www.youtube.com/watch?v=dOV3gGncGU8)

Title: Re: My HAl Rig
Post by: MagnusWootton on November 03, 2021, 07:14:32 am
Looks like you want to implement something like hard disc streamed texture-mapping, for a bigger memory store.    From my experience I found hard disc reads were 20x faster than hard disc writes,   writing is even worse!    So if you kept the writes down it would go faster,  but still the reads are only slowish too, but writing is even worse.

I had a cellular automata thing happening off the hard disc,  and thats what I found.
(Learning/writing is REALLY slow,   but recall is not as bad.)
Title: Re: My HAl Rig
Post by: frankinstien on November 03, 2021, 10:10:29 pm
Quote
Looks like you want to implement something like hard disc streamed texture-mapping, for a bigger memory store.

I only use hard disks as a last resort storage medium for things that aren't used too often. Everything else is Gen3 or Gen4 Nvme Nand SSDs which have much much faster read and write speeds than hard disks. But Optane is one fast persistent medium 170 - 360 nanoseconds compared to Dram 90 nanoseconds. The speedup is orders of magnitude faster than Gen4 Nvme SSDs!
Title: Re: My HAl Rig
Post by: MagnusWootton on November 03, 2021, 10:42:24 pm
Indeed my knowledge on storage is pretty dated,  must be completely different world these days.
Title: Re: My HAl Rig
Post by: LOCKSUIT on November 04, 2021, 05:04:49 pm
@frankinstien

wut...

What do you think is more efficient than a neural network or tree tries? How so? A database of plain text?....That is impossible to be good... There isn't a better way to compress data you're using (the program AKA neural network + code). How would you store then 100MBs of text as memories? My own algorithm needs to store (once I or if I finish coding it:) word offsetted snapshots like "[we walked fast down] the", "we [walked fast down the]", that is every ~5 words in the 100MB is needed, and uses a trie/tree to do that. It will also store parts by pointing at near-root nodes ex "we ate">"so much" instead of storing those 2 words again. You can also sort of compress the words in the tree too, so if your vocab has 3 lengthy words abcdfgghefg thegfffhhfghfhfj ewettwrfdhdh, then you store just 1, 2, 3, each 1 byte, then you can store sentences like 12111312 in the tree. You could compress the code or tree more kind of, though may be not worth it and cost speed then.

I'm not sure why you think storing a database of every ~5 words in 100MBs of text would be more efficient RAM-wise, that would cost the most RAM.

Like, if we have 100MBs of text, and store every ~5 words in it into a tree, it would actually cost now at least over a GB of RAM, but it improves speed by far, for a brain. Furthermore, if we fed GPT 40GBs of text, it's be at least 40GB in RAM then. It isn't though, it's 12GB RAM on GPU,
Title: Re: My HAl Rig
Post by: frankinstien on November 05, 2021, 12:51:16 am
@frankinstien

wut...

What do you think is more efficient than a neural network or tree tries? How so? A database of plain text?....That is impossible to be good... There isn't a better way to compress data you're using (the program AKA neural network + code). How would you store then 100MBs of text as memories? My own algorithm needs to store (once I or if I finish coding it:) word offsetted snapshots like "[we walked fast down] the", "we [walked fast down the]", that is every ~5 words in the 100MB is needed, and uses a trie/tree to do that. It will also store parts by pointing at near-root nodes ex "we ate">"so much" instead of storing those 2 words again. You can also sort of compress the words in the tree too, so if your vocab has 3 lengthy words abcdfgghefg thegfffhhfghfhfj ewettwrfdhdh, then you store just 1, 2, 3, each 1 byte, then you can store sentences like 12111312 in the tree. You could compress the code or tree more kind of, though may be not worth it and cost speed then.

I'm not sure why you think storing a database of every ~5 words in 100MBs of text would be more efficient RAM-wise, that would cost the most RAM.

Like, if we have 100MBs of text, and store every ~5 words in it into a tree, it would actually cost now at least over a GB of RAM, but it improves speed by far, for a brain. Furthermore, if we fed GPT 40GBs of text, it's be at least 40GB in RAM then. It isn't though, it's 12GB RAM on GPU,

Compressed code? I remember in 2010 Space Oddessy Two, they were talking about HAL's memory as something called a non-linear worm thing or something of a jigama. Which was eluding to AI would need a non-linear memory to be functional. Clarke, I'm guessing, wasn't aware of Hashcodes and how they can speed up lookups, but Clarke would differently roll over in his grave if he knew that current AI has to iterate through the entire matrix of billions to get an output! I don't compress data or use binary trees, I use hashcodes to represent everything! So by breaking all pieces of data into hashcodes I can search for relations within a few computations and parallelize those as stimuli are inputted into the system. I can find partial features of words not as text but as definitions of classifications, functional processes, and/or episodic memory that relate to other data.  Those concepts are learned instantly, it's not a re-training of 700GB of data. I use a descriptive model(which I've posted on this board several times) which can incorporate any kind of data and encode that processes that use that data as boolean or spike code logic (read my post on Neurons are wrappers for digital processes.) that direct things like workflows or can be used in recursive analysis. Also, the timing chunker (read my post on the timing chunker that is modeled after human brain time chunking) uses hashcodes as well. Also remember that I use referencing, meaning that once I find one piece of data if it has relationships with other data, that data is referenced to the descriptor object(Review my object-oriented data model) so there is no searching for it, the data is available instantly! So with this approach, one piece of data can have relationships with millions of other data points, similar to neurons, and that data is readily available with a simple hash lookup.

So which is faster a room full of GPUs churning out iterations over billions of neurodes or a calculation of a hashcode done within a few hundred nanoseconds as I can incorporate SIMD and GPU to do hundreds of millions of these kinds of calculations. But the point is breaking stuff into nuggets of features that are computed into a hashcode will beat any ANN, even an army of ANNs in a warehouse of GPUs when it comes to finding data.
Title: Re: My HAl Rig
Post by: MagnusWootton on November 05, 2021, 04:35:15 pm
Clarke would differently roll over in his grave if he knew that current AI has to iterate through the entire matrix of billions to get an output!

Theres one matrix entry per synapse of the ANN,  so you have to run the synapses linearly.    That's to be expected, its not too much computation, In my mind, its perfectly feesable.

The bit you cant compute is working out what the synapse weights are.  and thats 2^synapses!   and thats why AGI isn't here yet.


I don't compress data or use binary trees, I use hashcodes to represent everything! So by breaking all pieces of data into hashcodes I can search for relations within a few computations and parallelize those as stimuli are inputted into the system. I can find partial features of words not as text but as definitions of classifications, functional processes, and/or episodic memory that relate to other data.  Those concepts are learned instantly, it's not a re-training of 700GB of data. I use a descriptive model(which I've posted on this board several times) which can incorporate any kind of data and encode that processes that use that data as boolean or spike code logic (read my post on Neurons are wrappers for digital processes.) that direct things like workflows or can be used in recursive analysis. Also, the timing chunker (read my post on the timing chunker that is modeled after human brain time chunking) uses hashcodes as well. Also remember that I use referencing, meaning that once I find one piece of data if it has relationships with other data, that data is referenced to the descriptor object(Review my object-oriented data model) so there is no searching for it, the data is available instantly! So with this approach, one piece of data can have relationships with millions of other data points, similar to neurons, and that data is readily available with a simple hash lookup.

So which is faster a room full of GPUs churning out iterations over billions of neurodes or a calculation of a hashcode done within a few hundred nanoseconds as I can incorporate SIMD and GPU to do hundreds of millions of these kinds of calculations. But the point is breaking stuff into nuggets of features that are computed into a hashcode will beat any ANN, even an army of ANNs in a warehouse of GPUs when it comes to finding data.

That sounds really cool!  you might be onto something amazing!
Title: Re: My HAl Rig
Post by: frankinstien on November 05, 2021, 10:59:24 pm
Just wanted to add that there are hashcode strategies where you can save quite a bit of CPU cycles where one side of the comparison can be done with as little as two instructions, so actually, the time can be in the tens of nanoseconds to do a look-up. Of course, mileage will vary according to coding approaches, OS environments, and CPUs or GPUs.
Title: Re: My HAl Rig
Post by: LOCKSUIT on November 06, 2021, 04:47:00 am
So you mean a sparse network then? Instead of computing the whole matrix. Ya, my AI actually would fit that bill then really good, if I can finish it. My network can come out to still small and fast as explained in my last post. Now, you say hashes, hmm, like the word 'walking' is converted into its ord and that chooses the location in a list in python code, hmm, yes that's very fast....but I bet the RAM will suffer let's see: You have 100MBs of text, and must store every ~4 words of it, and find them fast. To make the 100MBs hashable, you need to store in a small list (small as in 'using the hash table method') the ~4 words+prediction entailment, which means you need to store for 100MBs: 500MBs....40GBs?: 200GBs...So this extra large 40GB dataset of text that GPT has in RAM as 12GBs, would come to 200GBs needing to be in RAM. Or does it need be in RAM if is hashable (fast find) ? Anybody know? So for 10GBs of text it would work o-k, 50GBs of RAM needed then.

But don't forget you need to find ALL matches of "[we [walked [down [the]]]] ?___?", and combine the predictions to get a set of predicted words for all 4 matches, yup so 'the' has nearly 800,000 matches in 100MBs of wiki LOL, when they could all be put into a tree with max 50K vocab. You also need a semantic web like word2vec and need to store those embeds or connections.

So it's big....and slow...
Title: Re: My HAl Rig
Post by: frankinstien on November 06, 2021, 08:40:37 pm
So you mean a sparse network then? Instead of computing the whole matrix. Ya, my AI actually would fit that bill then really good, if I can finish it. My network can come out to still small and fast as explained in my last post. Now, you say hashes, hmm, like the word 'walking' is converted into its ord and that chooses the location in a list in python code, hmm, yes that's very fast....but I bet the RAM will suffer let's see: You have 100MBs of text, and must store every ~4 words of it, and find them fast. To make the 100MBs hashable, you need to store in a small list (small as in 'using the hash table method') the ~4 words+prediction entailment, which means you need to store for 100MBs: 500MBs....40GBs?: 200GBs...So this extra large 40GB dataset of text that GPT has in RAM as 12GBs, would come to 200GBs needing to be in RAM. Or does it need be in RAM if is hashable (fast find) ? Anybody know? So for 10GBs of text it would work o-k, 50GBs of RAM needed then.

But don't forget you need to find ALL matches of "[we [walked [down [the]]]] ?___?", and combine the predictions to get a set of predicted words for all 4 matches, yup so 'the' has nearly 800,000 matches in 100MBs of wiki LOL, when they could all be put into a tree with max 50K vocab. You also need a semantic web like word2vec and need to store those embeds or connections.

So it's big....and slow...

I don't know where you got your numbers from but certainly not from an understanding of hashcodes. So, take a word that has four characters with simple ASCII that's 4 Bytes, but a 32-bit hashcode is only 4 bytes and the word like "pneumonoultramicroscopicsilicovolcanoconiosis" is 45 characters(45 Bytes) but the word is represented by other components which means there are even more bytes involved where each word is stored with its OL, all of that reduces to a 32-bit hashcode!  The ontological component and the descriptor component provide feature or property states for each word and there is only one instance of those structures for each word.  I have something like 790,000 words stored and the OL database and it's only 391MB, but its hashcode store is only 3.2MB with 32-bit and 6.4MB with 64-bit codes! My older server has 128GB and the new system has 256GB. The 391MB with the addition 3.2MB is but a drop in the bucket of all the ram I have! The descriptor component is just starting out but right now is averaging 20,000 bytes per word, at 790,000 words that's 16GB to cache it, but its hashcode per word reduces to just 4 to 8 bytes!

Ok, so you might argue; but you have to index those features as well, and you're right, the current descriptor DB feature index averages 881 bytes per word, so 790,000 words would be just 700MB, where each feature is a single instance with a HashSet that stores a reference to the descriptor component, again a drop in the bucket of all the ram I have! So, even as the data grows as the system learns and makes those descriptor components more complex there is plenty of room and the hashcode burden is trivial.

If you remember my post of the time chunking scheme where I ran out of 128GB in 32 minutes, but I later simply stored threshold deltas of stimuli which kept everything manageable for days, where, yes eventually you'll need to manage the temporal resources by writing to disk, in this case, is NVMe gen3 or 4 SSD. The approach makes things pretty responsive when having to find data on the disk, which is indexed with hashcodes and inter-file locations.

Now here's your problem with ANNs, you have to iterate through the entire network that doesn't really work in a way that can represent meaning as a point in memory. Your ANN distributes the description of words across the entire network which is why you have to iterate through the entire matrix to get an output. My approach doesn't, the data is focused into structures that have single instances that can even dynamically change in real-time, meaning the system can learn while it executes!  As stimuli are entered into the system it is converted into hashcode sets that look up the relevant data that is associated with functions or processes to respond. So I don't have to iterate through the entire dataset as the Anns do, and can change the associations to those structures instantly, no retraining of the entire system.  Also, remember accessing other data that relates to the descriptors is referenced whose instances reference other data. So, algorithmically I can gain access to data to provide more capabilities without having to randomly search for it since it's right there for the taking because of how relationships are linked/referenced, again speeding up processing and not having to iterate through billions of other neurodes that aren't really representative of what is need but you have to calculate their contribution to the output regardless.

With an Ann you can't just find the functional data points with a query, but with this approach, you can and that's why only a fraction of the computational horsepower is needed compared to an ANN.  Here's another advantage, I can still use ANNs but they are much much smaller because they are focused on the semantic interpretations of a query that's initiated from stimuli, whose generalized states can be evaluated into patterns. Realize the ANN is called only after the data is matched to the stimuli. So the problem domain is much smaller than what GPT3 does which tries to encode everything into a big ANN. Also, this approach isn't trapped into an ANN solution only, so it opens up the framework to a universe of solutions, e.g. genetic algorithms, differential equations, Bayesian inference, etc.
Title: Re: My HAl Rig
Post by: MagnusWootton on November 06, 2021, 10:10:34 pm
Nice thinking,  and Its cool watching you strive off onto the cutting edge.

But you cant defy the law of context.  As in if you lose context, you cannot get it back.   Maybe u are allowed to lose it?  But if u do, it can never return and its permanent!

Read this paper::
https://matt.might.net/articles/why-infinite-or-guaranteed-file-compression-is-impossible/

Dont give up tho in you find it doesnt work,   because your thinking is good,  its the same whether your coming up with something works or not,   and AGI does take a huge optimization like this!!!   So dont give up and keep looking for it,  and it is special,  but it has to not defy context, but its kinda the same thing,  works like magic for sure!
Title: Re: My HAl Rig
Post by: frankinstien on November 06, 2021, 10:36:43 pm
Nice thinking,  and Its cool watching you strive off onto the cutting edge.

But you cant defy the law of context.  As in if you lose context, you cannot get it back.   Maybe u are allowed to lose it?  But if u do, it can never return and its permanent!

Read this paper::
https://matt.might.net/articles/why-infinite-or-guaranteed-file-compression-is-impossible/

Dont give up tho in you find it doesnt work,   because your thinking is good,  its the same whether your coming up with something works or not,   and AGI does take a huge optimization like this!!!   So dont give up and keep looking for it,  and it is special,  but it has to not defy context, but its kinda the same thing,  works like magic for sure!

From a conceptual perspective maybe not. Remember, that when encoding things the more understanding of a concept on the decoder's side the more compression you can have. So let's say I use the set of symbols e=mc^2 that's only six characters but because I understand some physics and math the concepts of energy, mass, the speed of light and the mathematical function of squaring don't have to be included in the data encoding! So, too with this approach, because the context is a concept it resolves to a set of descriptors already stored, so any page, paragraph, or sentence or sensory stimuli computes to a set of concepts already stored. New concepts or variances of concepts can be helped by adding attributes from existing descriptors or borrowing from existing descriptors through nesting. Such contextual artifacts can then be shared across many other data points! So, we prevent duplication of data by insuring single instances of concepts that can be referenced at any time.
Title: Re: My HAl Rig
Post by: MagnusWootton on November 06, 2021, 10:47:09 pm
From a conceptual perspective maybe not. Remember, that when encoding things the more understanding of a concept on the decoder's side the more compression you can have. So let's say I use the set of symbols e=mc^2 that's only six characters but because I understand some physics and math the concepts of energy, mass, the speed of light and the mathematical function of squaring don't have to be included in the data encoding! So, too with this approach, because the context is a concept it resolves to a set of descriptors already stored, so any page, paragraph, or sentence or sensory stimuli computes to a set of concepts already stored. New concepts or variances of concepts can be dealt with by adding attributes from existing descriptors or borrowing from existing descriptors through nesting. Such contextual artifacts can then be shared across many other data points! So, we prevent duplication of data by insuring single instances of concepts that can be referenced at any time.

If you arent losing context,  I have no problem with what ur doing, it could definitely work.    On the internet many of these so called "quantum people" are talking about "breaking math" all the time, and I disagree with it,  maths is invincible it cannot be broken.    the square root of -1,  some would say, its "breaking math"  But to me it isnt, its how it is supposed to be,   If anyone ever defies the law of context I doubt it to the fullest extent.    If mathematics didnt make sense for us, our lives are even more foolish than they are already.

So I doubt its "breakable",  mathematics and the logic that makes it is unbreakable.
Title: Re: My HAl Rig
Post by: frankinstien on November 07, 2021, 08:27:33 pm
I have found favor from the AI GPU gods,   :party_2: I actually picked up a NVIDIA GeForce RTX 3080 10GB for $750!  :dazzler:
 
The RTX 3080 has a whopping 238 TFLOPS of Int8 (https://hothardware.com/reviews/nvidia-geforce-rtx-3080-ampere-gpu-review).  :35:

There's about a 13% difference between the RTX 3080 and RTX 3090. Now having both a Vega 56 and an RTX 3080 on the same machine should be interesting.

Title: Re: My HAl Rig
Post by: LOCKSUIT on November 08, 2021, 03:04:51 am
So you mean a sparse network then? Instead of computing the whole matrix. Ya, my AI actually would fit that bill then really good, if I can finish it. My network can come out to still small and fast as explained in my last post. Now, you say hashes, hmm, like the word 'walking' is converted into its ord and that chooses the location in a list in python code, hmm, yes that's very fast....but I bet the RAM will suffer let's see: You have 100MBs of text, and must store every ~4 words of it, and find them fast. To make the 100MBs hashable, you need to store in a small list (small as in 'using the hash table method') the ~4 words+prediction entailment, which means you need to store for 100MBs: 500MBs....40GBs?: 200GBs...So this extra large 40GB dataset of text that GPT has in RAM as 12GBs, would come to 200GBs needing to be in RAM. Or does it need be in RAM if is hashable (fast find) ? Anybody know? So for 10GBs of text it would work o-k, 50GBs of RAM needed then.

But don't forget you need to find ALL matches of "[we [walked [down [the]]]] ?___?", and combine the predictions to get a set of predicted words for all 4 matches, yup so 'the' has nearly 800,000 matches in 100MBs of wiki LOL, when they could all be put into a tree with max 50K vocab. You also need a semantic web like word2vec and need to store those embeds or connections.

So it's big....and slow...

I don't know where you got your numbers from but certainly not from an understanding of hashcodes. So, take a word that has four characters with simple ASCII that's 4 Bytes, but a 32-bit hashcode is only 4 bytes and the word like "pneumonoultramicroscopicsilicovolcanoconiosis" is 45 characters(45 Bytes) but the word is represented by other components which means there are even more bytes involved where each word is stored with its OL, all of that reduces to a 32-bit hashcode!  The ontological component and the descriptor component provide feature or property states for each word and there is only one instance of those structures for each word.  I have something like 790,000 words stored and the OL database and it's only 391MB, but its hashcode store is only 3.2MB with 32-bit and 6.4MB with 64-bit codes! My older server has 128GB and the new system has 256GB. The 391MB with the addition 3.2MB is but a drop in the bucket of all the ram I have! The descriptor component is just starting out but right now is averaging 20,000 bytes per word, at 790,000 words that's 16GB to cache it, but its hashcode per word reduces to just 4 to 8 bytes!

Ok, so you might argue; but you have to index those features as well, and you're right, the current descriptor DB feature index averages 881 bytes per word, so 790,000 words would be just 700MB, where each feature is a single instance with a HashSet that stores a reference to the descriptor component, again a drop in the bucket of all the ram I have! So, even as the data grows as the system learns and makes those descriptor components more complex there is plenty of room and the hashcode burden is trivial.

If you remember my post of the time chunking scheme where I ran out of 128GB in 32 minutes, but I later simply stored threshold deltas of stimuli which kept everything manageable for days, where, yes eventually you'll need to manage the temporal resources by writing to disk, in this case, is NVMe gen3 or 4 SSD. The approach makes things pretty responsive when having to find data on the disk, which is indexed with hashcodes and inter-file locations.

Now here's your problem with ANNs, you have to iterate through the entire network that doesn't really work in a way that can represent meaning as a point in memory. Your ANN distributes the description of words across the entire network which is why you have to iterate through the entire matrix to get an output. My approach doesn't, the data is focused into structures that have single instances that can even dynamically change in real-time, meaning the system can learn while it executes!  As stimuli are entered into the system it is converted into hashcode sets that look up the relevant data that is associated with functions or processes to respond. So I don't have to iterate through the entire dataset as the Anns do, and can change the associations to those structures instantly, no retraining of the entire system.  Also, remember accessing other data that relates to the descriptors is referenced whose instances reference other data. So, algorithmically I can gain access to data to provide more capabilities without having to randomly search for it since it's right there for the taking because of how relationships are linked/referenced, again speeding up processing and not having to iterate through billions of other neurodes that aren't really representative of what is need but you have to calculate their contribution to the output regardless.

With an Ann you can't just find the functional data points with a query, but with this approach, you can and that's why only a fraction of the computational horsepower is needed compared to an ANN.  Here's another advantage, I can still use ANNs but they are much much smaller because they are focused on the semantic interpretations of a query that's initiated from stimuli, whose generalized states can be evaluated into patterns. Realize the ANN is called only after the data is matched to the stimuli. So the problem domain is much smaller than what GPT3 does which tries to encode everything into a big ANN. Also, this approach isn't trapped into an ANN solution only, so it opens up the framework to a universe of solutions, e.g. genetic algorithms, differential equations, Bayesian inference, etc.

Let me try again: You have to understand that when it comes to GPT and my AI, that if you want to attain the same level of results (unless you've found some more efficient way and implemented it and can show it works (others tell me to, so I say it back: show me code!)), then you need to store every ~4 word long strings in 40GBs of text, basically. This allows you to take a prompt like 'the>___' and predict the next word properly, knowing what word is usually the word that comes next. Blending methods like this brings All the magic, it is far from copying the dataset. But see, in 40GBs of text, 'the' appears lots, just 100MB has 800,000 occurrences. So you need to put those all into a trie tree, otherwise you'll be matching all them every time you are given that sentence to complete. So you may predict now: the > cat/ home/ arm/ light/ throw/ moon.....and maybe dog was seen more than moon, so you more heavily predict dog then.
Title: Re: My HAl Rig
Post by: frankinstien on November 08, 2021, 06:49:05 am
Quote
But see, in 40GBs of text, 'the' appears lots, just 100MB has 800,000 occurrences. So you need to put those all into a trie tree, otherwise, you'll be matching all of them every time you are given that sentence to complete. So you may predict now: the > cat/ home/ arm/ light/ throw/ moon.....and maybe dog was seen more than moon, so you more heavily predict dog then.

No, I don't use a trie. I have posted many times the descriptor concept on this forum, but I'll do it again to clarify some things. You appear to think that I have duplicate data into tiers for each sentence but I don't. The word "the" is a determiner and is handled by the NLP, it does have a representation in the OL but its state as a determiner suffices as a state in itself. While a word like "human" can appear many times in text there is only one instance of "human" as a descriptor and other classifications that are coded in an ontological framework. Since only one instance of a word is allowed in the database any contextual variances are considered concepts that are symbolically represented by a word or set of words as a term. Such concepts get associated with words through descriptor objects and OL hierarchies, as shown below:

Note: hover over an image and click on it to get a bigger image.

(https://i.imgur.com/J3aAjDw.jpg) (https://i.imgur.com/J3aAjDw.jpg)

The OL will have the concept of human across many groupings or hierarchies but there is only one instance of Human that all those headings reference.

(https://i.imgur.com/VBDPZRD.png) (https://i.imgur.com/VBDPZRD.png)

Now as I mentioned earlier under this thread; If something is already known then it doesn't have to be duplicated, so to remove redundant data inheritance or nesting is used, as shown below:

(https://i.imgur.com/J3aAjDw.jpg) (https://i.imgur.com/J3aAjDw.jpg)

As you can see "Human" inherits from the concept of "Animal":

(https://i.imgur.com/WeWLPXU.png) (https://i.imgur.com/WeWLPXU.png)

Animal has many other concepts such as Head, Neck, Torso, etc. Those concepts are nested into Animal, rather than inheriting from them, since those properties are parts of an animal, they need to be exposed as such. Those descriptors have vector definitions that are both numeric and text, where the enumerator is used in algorithms.  Below is an image of a vector state of a word in a descriptor object.

(https://i.imgur.com/E5wYYly.png) (https://i.imgur.com/E5wYYly.png)

Now, on my web site blog (https://wraithbots.com/2018/11/26/predicting-vs-reacting/) I have a write-up on whether to predict or react and many times the ability to react proves far better a strategy than predicting! Now, let's look at a sentence parsed for its grammar:

(https://i.imgur.com/41bg9T4.jpg) (https://i.imgur.com/41bg9T4.jpg)

The parser groups the words into noun phrases, verb phrases, prepositional phrases, etc, and identify their parts of speech. That information is volatile, once the sentence, paragraph, or page has been evaluated by other logic it's disposed of!  Well sort of, its episodic representation is stored as text along with the logical interpretation of it. The reason for that involves using memory as humans do where remembering events is actually re-evaluating with new perceptions. Now the descriptors and OL hierarchies along with the NLP's output help correlate what a sentence means reactively, not through some kind of stepwise prediction that wastes CPU cycles on wrong predictions. Because the NLP segments the sentence it's possible to compute relationships based on concepts that those words relate to. This also allows for the machine to learn by finding similarities and/or asking a mentor questions about a word or some segment of the sentence or the entire sentence.

OK, so now you should be able to see that the text is turned into meaningful vectors without having to apply an ANN and those sentences are not stored, now to find the correlations mention above, I could use an ANN, but it's a pared-down ANN where the OL hierarchy can constrain the problem domain and I apply that network as needed instead of needlessly computing neurodes that have very little contribution to the output but have to be computed none the less.
Title: Re: My HAl Rig
Post by: LOCKSUIT on November 08, 2021, 06:43:19 pm
But does it predict as good as GPT-1? / Show me it running. Ideas don't mean much unless you have a theory I can grasp "quickly in 1 post".

Yes, I see your post, Reflexes are great, but Prediction is greater and general as shown in GPT-3. I rarely use my primitive reflexes (~20 I was born with). And learning to walk won't solve cancer or think about 'a wide range of situations'. Capturing data skips 'body learning' and puts you into a Simulation mentally, which is safer and faster etc. It's way better than even a computer sim, because predicting like DALL-E is cheaper than running a real sim with fluid etc.

I see you have a database that uses nesting and inheritance. You could probably let an AI learn those for you instead of writing them all in. It'd have to be vision, and use word2vec or such. So far so good. But the biggest problem is I don't think it can predict the rest of an image or sentence like GPT/ Jukebox/ DALL-E can. I was given lengthy exactly_human_level techno completions from Jukebox.

The reason a trie tree/ network needs to store 'the', 'but the', 'and the', 'cat and the', 'move and the', 'wind move and the', etc, is because it is storing the different contexts in which 'the' appears in. The more words and their order in the context that matc, the better for prediction. It then uses the longest matches to retrieve a prediction of the next word, and combines shorter predictions when its long matches have few experiences (usually is the case) in what word Usually comes next. It also can use robustness to holes and delay ex. 12r45>6, 12r345>6, 1 2 3 4 5 > 6.

NARS usage Guide (short read)......isn't natural language input/output, but rather Narsese! Apparently they have or are thinking about a way to make it natural input/output, hence the whole Narsese (logic) thing must be just "GPT statistics" and not logic/rule based AI.
https://www.google.com/url?sa=j&url=https%3A%2F%2Fcis.temple.edu%2F~pwang%2FImplementation%2FNAL%2FNAL-Guide.html&uct=1593766665&usg=Aq4qY4OLx-WK_HAKqmOG4qyU35k.

If you think logic based AI like NARS makes sense because it tries to use less data/ resources and more intelligently look at the context with "constraints/ rules" to predict the next word, you are mistaking some things... NARS [may] have some good ideas, but they should be able to be added, and Should, to GPT. How? I am not sure what NARS does, but I have a feeling it needs humans to write in Properties of Things, Relationships, Verbs, etc, (in Narsese and not natural language), which is too much work to be practical, then it can say "I bought a ___" where it can be anything because when you say bought, it can be anything, but when you say "a snake ", it can't be anything really usually, "a snake bit me" is ok, "a snake gift sold" is a bit uncommon, "a snake car"....so bought> allows more possible things to follow, other words don't, some require even very similar matches look: A similar word to car is: truck/ van/ vehicle/ etc. A different word than car is: ---anything---, and so there is how that mechanism works. It can 'learn' this, and predict for unseen words that they too probably can go there if most other words do, or most other felines do.
Title: Re: My HAl Rig
Post by: frankinstien on November 08, 2021, 09:09:02 pm
Quote
If you think logic based AI like NARS makes sense because it tries to use less data/ resources and more intelligently look at the context with "constraints/ rules" to predict the next word, you are mistaking some things... NARS [may] have some good ideas, but they should be able to be added, and Should, to GPT. How? I am not sure what NARS does, but I have a feeling it needs humans to write in Properties of Things, Relationships, Verbs, etc, (in Narsese and not natural language), which is too much work to be practical, then it can say "I bought a ___" where it can be anything because when you say bought, it can be anything, but when you say "a snake ", it can't be anything really usually, "a snake bit me" is ok, "a snake gift sold" is a bit uncommon, "a snake car"....so bought> allows more possible things to follow, other words don't, some require even very similar matches look: A similar word to car is: truck/ van/ vehicle/ etc. A different word than car is: ---anything---, and so there is how that mechanism works. It can 'learn' this, and predict for unseen words that they too probably can go there if most other words do, or most other felines do.

You're looking at this problem from the ANN's regression where it's looking at patterns of text and associating them with some term or response. That's not how the symbolic approach works. When you read you don't predict you evaluate what is being said. So, humans don't need an example of "Snake car" because we can evaluate whether the term "Snake" is a noun or an adjective that describes a car.  Now, if you've never heard of a "Snake car" you'd ask what is a "Snake car" , with an ANN it can't ask what a Snake car is, but a symbolic approach can. The symbolic approach can learn immediately and without having to re-train on a gazillion GBs of data. 8)
Title: Re: My HAl Rig
Post by: LOCKSUIT on November 08, 2021, 09:23:51 pm
Umm....an Ann can ask what a snake car is.....

You need to take ex. GPT, and make it usually predict (using favoring force) "ask what is X" if it predicts (after recognition) "I'm in-confident at predicting the next word".

You give it context, it predicts what is common or favorite (food/ ask twice!/ computers), then it predicts again further.

No not retrain, it's called fine-tune (continues training), and prompt engineering (topic: poem, write like <title> newline <poem briefing> etc), and goal learning (learns to predict 'ASI' all day now, new goal, always on mind!). That controls the whole model.
Title: Re: My HAl Rig
Post by: frankinstien on November 08, 2021, 09:48:20 pm
Umm....an Ann can ask what a snake car is.....

You need to take ex. GPT, and make it usually predict (using favoring force) "ask what is X" if it predicts (after recognition) "I'm in-confident at predicting the next word".

You give it context, it predicts what is common or favorite (food/ ask twice!/ computers), then it predicts again further.

No not retrain, it's called fine-tune (continues training), and prompt engineering (topic: poem, write like <title> newline <poem briefing> etc), and goal learning (learns to predict 'ASI' all day now, new goal, always on mind!). That controls the whole model.

Seriously, you compare fine tuning (https://beta.openai.com/docs/guides/fine-tuning) to the immediate learning step that a symbolic approach can do? Also, you don't have to pay  to teach the symbolic approach.

And yet again, if you look at the page I referenced, because of the huge amount of resources used by GTP3 you always have to check if you're abiding by OpenAI's rules...
Title: Re: My HAl Rig
Post by: LOCKSUIT on November 08, 2021, 10:06:34 pm
The Persona that Facebook's Blender uses, which is also called taming/ forcing/ controlling the model, can immediately make it strongly favor to predict something new.

As for getting a new memory 'in-to' the network, i.e. GPT uses Backprop, it may be a bit hard to make it 'store' something new without forgetting other memories or whatever, maybe, I'm unsure, but if we used a trie tree, you could quickly just store exactly just the new memory, or relationship 'snake car' similar to 'car' by ex. 70%.



Anyway, not saying your way won't work, just, I don't know what your way is maybe... Can you explain "how" your AI can take a natural prompt (human English sentence question, or image), and generate the rest of that image or sentence? Like, where does it get the next word from, and how does it know that 'we>ate' is more likely than 'we>floored'? How does it deal with 1234567> and 12th567> and 1 2 3 4 5 6 7 ? and 12567> ? Mine uses delay and hole matching, and merges all matches to get 1 set of predicted words, softmaxed (each word how much % of 100% is predicted).

What are the rules in your AI? Like give me an answer like this: nests, inheritance, hierarchy, goals, invariant spots (we bought <anything goes here it learnt> today), exponential function, multi-sensory, human in the loop (i dislike these AIs), etc. So I can understand the parts of your AI then "narrow down on them and see em all easy".
Title: Re: My HAl Rig
Post by: frankinstien on November 08, 2021, 11:49:47 pm
The approach for this is not finding text patterns and applying what seems to be the best fit, but to evaluate the logic of a sentence and compute a logical response. So you're examples of 1234567, 123th67, 1 2 3 4 5 6 7 would be handled as follows:

(https://i.imgur.com/MUVQN1P.jpg) (https://i.imgur.com/MUVQN1P.jpg)

So notice that the NLP broke the sequence into two numbers, if you look up at the part of speech those items resolve to is CD(cardinal number). So the NLP saw a break in the sequence when it saw t, which is not a CD. As it evaluated th67 it realized that there is no word within that sequence so it labeled it a form of number. When the logic starts to evaluate the sentence's meaning it will validate whether th67 is actually a number, which it isn't, and will ask for clarification from a mentor. Note further down the sentence it labels 1234567 as a single number.

Now look at this image:

(https://i.imgur.com/CVNtD8i.jpg) (https://i.imgur.com/CVNtD8i.jpg)

I replaced the 123th67 with 123main24 and notice that the NLP didn't break up the sequence because it found a real word, "main", so that could be an address with missing spaces or maybe not. Notice that it identified it as a cardinal number. The logic that evaluates the relationship between words will notice that the labeling of CD on the sequence is incorrect and this will prompt a correction to add spaces between the numbers and the word "main" because the wording of the sentence implies an address, if the sentence did not imply an address it then asks what is 123main24.

And here is the list of 1 2 3 4 5 6 7 where each number is what the NLP identified each one to be, a number.

(https://i.imgur.com/zV3NtnH.jpg) (https://i.imgur.com/zV3NtnH.jpg)

And to answer the question as how does it predict if needed, the approach does something similar and that is it anticipates a contingency based on its experience. There is also partial feature fit where it can associate with an image or audio that it has heard before and apply whatever algorithm(s) to process it. This includes lists of functions it has learned in the past to reach an idea goal.
Title: Re: My HAl Rig
Post by: LOCKSUIT on November 09, 2021, 04:57:06 pm
Are you trying to build AGI / human level AI?

If so, then how would your AI solve some problem like IDK, let's say this:

The rabbit was trapped in a glass jar and Jimmy saw this, and a lake of lava was in front of Jimmy too surrounding the rabbit. So to help him, Jimmy ___________________________________?
Title: Re: My HAl Rig
Post by: frankinstien on November 10, 2021, 03:06:56 am
Are you trying to build AGI / human level AI?

If so, then how would your AI solve some problem like IDK, let's say this:

The rabbit was trapped in a glass jar and Jimmy saw this, and a lake of lava was in front of Jimmy too surrounding the rabbit. So to help him, Jimmy ___________________________________?

The sentence has certain concepts that Amanda needs to learn first, and that is what is "lava". From that perspective, a descriptor object for lava needs to be created with vectors such as the class "threat" which includes labels like hot, cold, danger, injury, destruction, pain, etc. I am working on building a speech recognition interface to build descriptor objects from conversations. Again, because I can constrain the problem domain achieving something similar to what Google did to order a pizza over the phone is possible to build descriptor objects. There also needs to be a sense of self  (https://aidreams.co.uk/forum/general-ai-discussion/modeling-consciousness/msg67355/#msg67355)that relates to qualities of emotional states. Then Amanda needs to learn to empathize and to what degree, which requires descriptors of empathy with emotional nestings (https://aidreams.co.uk/forum/general-project-discussion/emotions-15065/msg65510/#msg65510), and since the rabbit is surrounded by the lava then there may not be a means to rescue the rabbit. Then again if Amanda had been bitten by rabbits in the past there might not be much empathy for them.  >:D

You see, I'm not trying to build a chatbot, where it would answer from some scanning of pet owners posts about rabbits and how they would rescue the rabbit at all costs and say something like: "I'll think of something or call someone to help me rescue the rabbit." Which is what most would think is such a human-like response. I want to teach Amanda to be human (https://aidreams.co.uk/forum/general-project-discussion/), not to respond using a statistic of words that are usually associated with a topic.

So, I still have more work to do.  Right now Amanda just knows that a rabbit is a form of animal :)



Title: Re: My HAl Rig
Post by: LOCKSUIT on November 10, 2021, 05:36:53 pm
You want to build human level AI? AGI?


my solution BTW :) >>

The rabbit was trapped in a glass jar and Jimmy saw this, and a lake of lava was in front of Jimmy too surrounding the rabbit. So to help him, Jimmy took a big jump over the lake of lava, went up to the glass jar and twisted it as hard as he could, and grabbed the rabbit out of the glass jar.


I'm still very unsure how you can get around "GPT"/ my AI mechanisms, for example to answer that question above you need to heavily use pattern finding mechanisms, like taking the key words at least like trapped, rabbit, lava lake, jar, and their order strung together (sentence), and this is how you pull the next word (answer) basically..."I walked down the>street", so you can answer it even if see "i then actually walked so fast down some new long long>". Recency tells us maybe "long" should come next, but eventually stop too from boredom. I call this match recognition >then> prediction. I don't understand how your AI can get the next word to predict. I'd like if you could a full clear explanation of all mechanisms that change predictions it found for an unseen context/problem.
Title: Re: My HAl Rig
Post by: frankinstien on November 10, 2021, 07:47:44 pm
You want to build human level AI? AGI?


my solution BTW :) >>

The rabbit was trapped in a glass jar and Jimmy saw this, and a lake of lava was in front of Jimmy too surrounding the rabbit. So to help him, Jimmy took a big jump over the lake of lava, went up to the glass jar and twisted it as hard as he could, and grabbed the rabbit out of the glass jar.


I'm still very unsure how you can get around "GPT"/ my AI mechanisms, for example to answer that question above you need to heavily use pattern finding mechanisms, like taking the key words at least like trapped, rabbit, lava lake, jar, and their order strung together (sentence), and this is how you pull the next word (answer) basically..."I walked down the>street", so you can answer it even if see "i then actually walked so fast down some new long long>". Recency tells us maybe "long" should come next, but eventually stop too from boredom. I call this match recognition >then> prediction. I don't understand how your AI can get the next word to predict. I'd like if you could a full clear explanation of all mechanisms that change predictions it found for an unseen context/problem.

Well, what if the lava bank is too wide to jump over? Why didn't your solution ask the question: "How wide is the lava bank?" and/or "Whose rabbit is it anyway?" See, your AI has no self-awareness, it just responds with words that are the best fit from the statistics it collected. My approach uses word vectors to determine abilities, threats, etc and experience (Amanda needs to know that jumping has an efficacy which it can learn from conversations that depict various scenarios where jumping was involved), it applies vectors to verbs, adjectives, etc to determine logical relationships that imply contexts of action and relations to consequences, such as can Jimmy jump and is it worth jumping if there is too much pain if Jimmy fails to make it across.
Title: Re: My HAl Rig
Post by: LOCKSUIT on November 10, 2021, 09:15:02 pm
"Why didn't your solution ask the question: "How wide is the lava bank?" and/or "Whose rabbit is it anyway?" See, your AI has no self-awareness"

I think you have no self awareness lol, why didn't you answer my question at the top of my last post? 0>0 Or my how yours works question? X>X

My AI if finished, and GPT, can easily be trained on chat data or made to ask multiple questions before writing a sentence completion/ extension. That is very easy to add to GPT.


You can see below how DALL-E can be (in my plans) made to have motor executive control, and desires to think a certain way (RL), and to look around on notepad using motors and do tasks like https://openai.com/blog/grade-school-math/

>> something i had written:

My 1st question/theory was: If I predict/imagine a scene/movie like DALL-E, that already describes visually all of the expected motor actions naturally (of my plan, ex. to raise my arm up), all that's needed now is the limbs to act (activated is each limb in the image (its rotation and speed)) and the Cerebellum to correct numerous small errors so the input matches the desired target image in brain that is stored. No motor cortex is needed, sensory hierarchies store the same things. Only leaf 'limb nodes' are needed.

And but so my 2nd question was how can I think of the movie, and decide to do it or not do it. For example DALL-E may spit out a prediction "<a movie of raise hand> + DO IT !!", and so clearly it has a plan in mind and also is going to do it in real life. The problem with this theory though is I can think of the scene and predict the do_it and still withhold myself from acting it out in real life. I know it needs to think of a movie plan and I know it needs to predict the 'do it' memory, it can't just predict the movie plan and expect RL to handle deciding to do it or withhold itself, it must predict using sensory, because it is all context based and requires simply a prediction to decide to do it in real life, and using RL should be to control sensory prediction like Facebook's Blender chatbot (which is cooler than GPT because it uses word desires/goals, a forcing called Persona). I'm thinking now maybe my goal is strong enough that says to not_do_it, hence when I predict to do_it and I don't do it, I am actually not hearing it but in the background the weight is stronger still. To understand what I mean, see Facebook's Blender chatbot. It uses such, called Persona, forcing certain words in the background no matter if heard/said other words. So: it is against me no matter if I scream in my brain 'do it' constantly and in different ways ex. 'act it!', 'move!', 'initiate plans!'.
Title: Re: My HAl Rig
Post by: MagnusWootton on November 10, 2021, 11:12:30 pm
I'm a little different that I don't think I think like my Ai does,    I think its got huge intelligence defecits.    :2funny:
Title: Re: My HAl Rig
Post by: frankinstien on November 12, 2021, 02:26:48 am
Quote
using RL should be to control sensory prediction like Facebook's Blender chatbot (which is cooler than GPT because it uses word desires/goals, a forcing called Persona). I'm thinking now maybe my goal is strong enough that says to not_do_it, hence when I predict to do_it and I don't do it, I am actually not hearing it but in the background the weight is stronger still. To understand what I mean, see Facebook's Blender chatbot. It uses such, called Persona, forcing certain words in the background no matter if heard/said other words. So: it is against me no matter if I scream in my brain 'do it' constantly and in different ways ex. 'act it!', 'move!', 'initiate plans!'.

Really? Look at the videos below of the Blender Chatbot. The hype behind this stuff is pretty high. This reminds me of a tool I built for a company where everyone thought that because the tool can highlight a circuit trace on a schematic diagram and even continue the trace across all desperate pages that a circuit could be a part of, everyone thought the machine knew the connectedness of the components. But, in fact, it did not, it only drew a highlight on the circuit lines that connected the components. But that's all a human needs to get the impression that the machine has a higher level of intelligence. Get it?  These bots respond in complete sentences but they aren't really understanding what the person is saying or what they're saying in response. Looking at the video below the responses are what has been the norm for chatbots for the last decade. I mean in the first video the bot responds with "I have a crush on my coworker" Which we know the bot doesn't have, so it's just random gibberish that is baked just well enough that an ANN can apply it. The guy in the video then is elated that it responded the way it did, stating "It acts soo human!" Remember the schematic tool, same kind of thing here as well...

Oh, and I tried to install ParlAI, what a nightmare! The documentation is terrible. The install bombed because rust wasn't on my machine, but the documentation never mentioned that rust had to be installed. Then issues with Torch, not good so far. But in any case, looking at the videos the response time of this chatbot is terrible.

ParlAI (https://www.youtube.com/watch?v=wTIPGoHLw_8)

ParlAI (https://www.youtube.com/watch?v=uOdBEZd-bow)
Title: Re: My HAl Rig
Post by: LOCKSUIT on November 12, 2021, 05:10:00 am
You keep saying "baked in replies". This is far from how GPT works. GPT is literally the opposite, and that is why DALL-E as shown on openAI.com can complete the rest of an unseen image you hand it so good. Also Transformers achieve the best general-purpose context prediction scores, and it's no wonder. Also I know how to make an easy to explain architecture 'like' GPT and so I completely to the bones understand 'how' it solves unseen prompts.

No, nope, Blender is not a chatbot and doesn't just get told to say 'cars', it is GPT or the similar and when you tell it to predict more often 'cars', it will leak to similar words like 'trucks' and 'goods'. This controls the Whole model so it literally sounds like it is a car hobbyist. You can set its Goals/Personas as phrases too, or at least in theory obviously.

And this guy below went a step further and gave it a notepad diary. We are getting close to AGI now, if this was DALL-E (DALL-E is multi-sensory).

ebook:
https://www.barnesandnoble.com/w/natural-language-cognitive-architecture-david-shapiro/1139957470?ean=2940162202622

If you want a paperback version of the book you can buy it here:
https://www.barnesandnoble.com/w/natural-language-cognitive-architecture-david-shapiro/1139957470?ean=9781668513118

Title: Re: My HAl Rig
Post by: MagnusWootton on November 12, 2021, 11:56:39 am
Locky's right -   that if you just have a sentence  "cat jumped over the fence"   you could say the computer knows nothing about its just a string pattern right, but it actually still counts as a tiny bit of knowledge, thats even less than cat or fence or jump,   it only knows one little tiny thing, and thats just the string itself.

Why OPEN-AI calls it a transformer, IMO, is because its going to take this string, and try and get more out of it,  by putting it through processes, which make it more plastic and more useful for different uses/applications.

There is alot of information there,  but if u just chuck it in a markov chain, it doesnt count for much,  and thats how FRANKENSTEIN is right.   its not in a form where its useful yet,  unless u make it useful.