[LOCKSUIT]

for example to my above posts

it'd store:

"lobby at the old school and"
"lobby at the old school"
"lobby at the old"
"lobby at the"
"lobby at"
"lobby"
.......these are strange uncommon cuts from some sentence, plus, as it adds ONE word at a time, it's sure to add clutter and non-reuseable features.

[spydaz]

in that image, it does have more than 2/3 nodes, a structure, yet, it is a bad take from a sentence, it'll never be re-used, and yes it got re-used by the word parts that followed but, that's all it'll ever get! It'll ass tons of clutter and false real world knowledge....slow searching...etc!! For a AGI..

Also, That's also bad for re-use, again, it'll never be reusing those features...

- because it's doing the left side example, not the one on the right i show.

Again for a three word sentence ? what ngram actually contains the subject or the predicate or the object ? .... the answer is in the question as usual...
a) 1
b) 2
c) 3
d) 4

Everything depends on YOUR level of understanding....

[LOCKSUIT]

oh and I mean that that is taken from a larger sentence like this

.....................[...............]...........

. = word

it won't know what to combine together from a sentence....it'd do all combos lol

[spydaz]

for example to my above posts

it'd store:

"lobby at the old school and"
"lobby at the old school"
"lobby at the old"
"lobby at the"
"lobby at"
"lobby"
.......these are strange uncommon cuts from some sentence, plus, as it adds ONE word at a time, it's sure to add clutter and non-reuseable features.

This is easily done ? its backward parsing.... Recursively....i did hint at this in the diagram (ADDED ITERATIVLY) <<< Forward parsing....
Not really needed but should be a function to be added... but the first suffix contains all your sub suffixes (they are prefixes of the first suffix) ?

Good to ask.... but these i mentioned in my early text messages

[spydaz]

oh and I mean that that is taken from a larger sentence like this

.....................[...............]...........

. = word

it won't know what to combine together from a sentence....it'd do all combos lol

A term document matrix to the applied Suffixes will show you the most valuable suffixes.(using node depths)....essentially creating a probabilistic Bag of suffixes<<<< again the algorythm mentioned before<<<<

I know your wondering how to use the outputs as inputs to your Neural network... this should be a forum project for you..... using the bag of word model for the neural network.... (its out there)

I asked one of the lecturers why they always set project which had been done before.... She said "im sending them on a learning journey"

[LOCKSUIT]

You mean a frequency matrix?

[spydaz]

You mean a frequency matrix?

A term document matrix to the applied Suffixes will show you the most valuable suffixes.(using node depths)

A trie tree cannot hold the same suffix twice??
It may occur twice if that be the case you may choose to record its "position in the node" <You could use this as a "RULE" <<<
When designing your requirements for your developer its best to specify your desires as rules which can the be applied to your insertion strategy or evaluation Strategy or pre-processing strategy... this is why its good to brainstorm an idea.. to define your rules and requirements... when actual development takes place compromises may have to be made or even upgraded data structures maybe required to accommodate rules or strategies

there are many rules which we could define to determine the value of a suffix! as it would then be the value of the suffix that would be the important outcome. Which could then be selected based on suffix length/suffix position/ suffix contents for sentence generation or construction .... Statistically

Right now we should be using an inference engine based algorithm and a natural language processing methodology  based with machine learning / statistical prediction .... using rules and strategies to determine if patterns are relevant or irrelevant taking actions based on statistical and probabilistic historical models... ... we have generated and processed some random sentence into some valuable data for use in output functions from learned input functions..... Based on some input T  and some learning factor C a new output is generated E <<<<<<<<< sounds like AI

[LOCKSUIT]

you say depth = it is not silly pairs of words ?

but as shown in this image I drew, it has great depth but bad snippet from a larger sentence:

[LOCKSUIT]

also, what if it hears a word or word pair ex. "complex AGI" 1 or 3 times in the frequency measure, that doesn't mean to delete it, it doesn't mean it is common or not, or does it hmm, it may see that "we are" appears many times (448 times), while "are all" appears fewer times (30 times), and "lobby at the old" appears once um, not sure if that only keeps the common pair features of sentences...

[spydaz]

This is you tree added

The branches with more than 1 level contain data ... the most data is contained in the branches with the highest nodes....
A pattern after testing multiple sentence shapes and adding into single trees may show that 3ngram sentences generally contain Determiner,Noun phrase... etc... such pattern would be learned over time if that was one of the objectives that the strategies were trying to fulfil.

The strategies bring you closer to your objective... even strategies used can also be measured and strategies can be deemed irrelevant as well ranking the strategies by their application can reduce code execution  speed therefore increasing the speed of "Thinking time"..... A potential objective of the algorithm to improve itself...refining it processes.... this may be applied to document processing learning or total learning process. (refinement)... reinforcement learning.... choosing winning strategies for recognised opponents ...applying a basic wining strategies or learning process to learn the opponents strategies and create a combination of strategies from existing rulest to win

[LOCKSUIT]

hmm you say frequency measure and depth nodes, well, that may mean "the" appears 100 times, "the cat" appears 50 times, "the cat sat" appears 20 times, and "the cat sat but" appears 2 times. But this happens for ALL bigger features like any darn sentence ex. "I work on AI all night but went skydiving" only appears once or so in meaning-wise, same for that weird sentence built up from a larger sentence "lobby etc etc", and no not for short things ex. "the" will be common and even the stupid uncommon "are then" will appear more yet less, maybe we should compare it to its peers ex. "the object" appears 444 times while "are then" appears 35 times, not sure that proves its good to trash though. So, frequency+depth nodes um, maybe combine the 2 peer reviews!?:

the object - 400 times
lobby at - 40 times
+
the object is so cool - 40 times
lobby at the old - 2 times
=
440
42

[LOCKSUIT]

That's cool spydaz. Yes I realized the first dotted line going down is a new start at each word ex. you get "at the old school". Hmm, but this is saving all the possible combos...sure it'll end up saying:


lobby at the old - 1 times
lobby at the - 5 times
lobby at - 7 times
lobby - 22 times

at the old - 20 times
at the - 300 times
at - 400 times

the old - 40 times
the - 400 times

old - 100 times


but um, hmm let's try adding the sum of frequencies in these depth nodes....calculating/thinking.........






Ok so the algorithm could see that old and the old and at the old and lobby at the old add up to:
161 times

features containing the:
766 times

at:
733 times

lobby:
35 times

the old:
61 times

at the:
326 times

at the old:
21 times

lobby at:
13 times

lobby at the:
6 times

lobby at the old:
1 times

[spydaz]

This is you tree added

The branches with more than 1 level contain data ... the most data is contained in the branches with the highest nodes....
A pattern after testing multiple sentence shapes and adding into single trees may show that 3ngram sentences generally contain Determiner,Noun phrase... etc... such pattern would be learned over time if that was one of the objectives that the strategies were trying to fulfil.

The strategies bring you closer to your objective... even strategies used can also be measured and strategies can be deemed irrelevant as well ranking the strategies by their application can reduce code execution  speed therefore increasing the speed of "Thinking time"..... A potential objective of the algorithm to improve itself...refining it processes.... this may be applied to document processing learning or total learning process. (refinement)... reinforcement learning.... choosing winning strategies for recognised opponents ...applying a basic wining strategies or learning process to learn the opponents strategies and create a combination of strategies from existing rulest to win

Therefore only a jedi can defeat another jedi....Only an AI could defeat another AI
Using the same strategy essentially finding a strategies to defeat itself.... em AI WAR

[LOCKSUIT]

see my edit above XD

ok so then re-arranging those numbers in order of highest to smallest frequency we get:



the:
766 times

at:
733 times

at the:
326 times

old:
161 times

the old:
61 times

lobby:
35 times

at the old:
21 times

lobby at:
13 times

lobby at the:
6 times

lobby at the old:
1 times

[LOCKSUIT]

and if we look again at:


lobby at the old - 1 times
lobby at the - 5 times
lobby at - 7 times
lobby - 22 times

at the old - 20 times
at the - 300 times
at - 400 times

the old - 40 times
the - 400 times

old - 100 times


we can see that in comparison to the add-ups, well,:
at the old:
21 times

lobby at:
13 times

lobby at the:
6 times

lobby at the old:
1 times

are pretty low common frequency


Hmm maybe I did this wrong..........................the single words are good ok but I have the in more pairs than lobby.....
I think we can conclude though that.......um............

.........But wait no I said earlier to compare small pair to small pair and big pair to big pair....obviously if we did that we'd see similar number of lobby and hat or tiger, but less of lobby at the old and more of the red person.

Hence some things are just not common, and but may be good, algorithm no worky...