Ai Dreams Forum

Member's Experiments & Projects => AI Programming => Topic started by: LOCKSUIT on February 01, 2020, 03:25:20 AM

Title: Possibly a breakthrough programming language?
Post by: LOCKSUIT on February 01, 2020, 03:25:20 AM
I came up with my own programming language. At least I think. Here it is below. It can do lots with text and even modify its own code. See how it grabs its pointer code and creates a node tree of pointers?

For comparison, C++ and the tutorials look much longer/harder https://www.learncpp.com/

In my language, all you code is: where to look (which function, or where in a string), and what to read/write. You can grab any letter in the code or even an input file and place it to write code/file. Similar to a Turing Tape machine, but very usable it seems.

g=goto
f=function
s=string
r=read
w=write
"""=any chars
###=digit
#=#
+=+1
-=-1
c=create function below
d=delete
OR
AND

REAL EXAMPLE:

gf3 c"""
gf3 gs3 gf4 gs3
5 gf3 r gs+ gf4 gs+ w
"""the cat was
gf0 r gf7 w gf0 gs3 gf6 w
OR
hmm...you get the idea

What I did above is goto line 4, create a line line5 below it starting with """. Then now I'm done line1 code I move onto line 2 and I goto line 5 we created and go to letter 4 in that line and then i just made the current position here now i'm gonna go to line 5 and go to letter 4. Now on code line 3 it says 5, so 5 loops. You can see a read/grab/clone something and drop it off somewhere. The gs+ just increments the position forward by 1.

Looking for feedback. Could it work?
Title: Re: Possibly a breakthrough programming language?
Post by: Hopefully Something on February 01, 2020, 04:14:59 AM
That's way more work than just learning a language, I'd like an auto translator from English to your existing code of choice. Maybe GPT2 could learn that from lots of examples or something. But  you'd need a program that is forgiving with errors. Or at least does a code spellcheck like Word.
Title: Re: Possibly a breakthrough programming language?
Post by: LOCKSUIT on February 01, 2020, 04:29:58 AM
No, it's not way more work. I fully understand it and can create my whole project in just 20 lines. My only question is if it will work. My input file is 100MB and must be fast/low RAM, and does many text manipulation stuff (which is always get this here, move this here, as I said in opening post......)

Hope someone who knows how to program can weigh in.

I'm checking this out, again.
https://developers.google.com/blockly
Title: Re: Possibly a breakthrough programming language?
Post by: LOCKSUIT on February 01, 2020, 04:43:50 AM
I edited the code description below the example a bit, made a mistake.
Title: Re: Possibly a breakthrough programming language?
Post by: WriterOfMinds on February 01, 2020, 05:43:13 AM
If this is the type of thing you're after, maybe you should learn sed.  https://en.wikipedia.org/wiki/Sed
Title: Re: Possibly a breakthrough programming language?
Post by: LOCKSUIT on February 01, 2020, 05:53:31 AM
https://www.gnu.org/software/sed/manual/sed.html

let me see...

something already seems off at first glance, :-)

Edit: Nope, reads like puddycake to me. Not simple enough.
Title: Re: Possibly a breakthrough programming language?
Post by: LOCKSUIT on February 01, 2020, 06:46:33 AM
Oh mine would need math functions like == and > but it seems all simple. I really don't get the whole need for the [j++ /2!%$## /n B! >>[j++.......
Title: Re: Possibly a breakthrough programming language?
Post by: LOCKSUIT on February 01, 2020, 06:49:56 AM
they say Blocky, above, by Google is not a toy. Maybe it grew up finally.

oh and:
https://www.youtube.com/watch?v=lPVJjQbEeN0
Title: Re: Possibly a breakthrough programming language?
Post by: LOCKSUIT on February 01, 2020, 07:08:45 PM
So far Blockly is working for me, I'll update in a few hours.

I'm coding like "woosh" right now hehe

"woosh"

IT'S WORKING!!!!!!! be right back
Title: Re: Possibly a breakthrough programming language?
Post by: LOCKSUIT on February 01, 2020, 07:46:07 PM
what i made so far

https://www.youtube.com/watch?v=eYmJ5FM7jbs
Title: Re: Possibly a breakthrough programming language?
Post by: Dat D on February 01, 2020, 07:48:46 PM
...
g=goto
f=function
s=string
r=read
w=write
"""=any chars
###=digit
#=#
+=+1
-=-1
c=create function below
d=delete
OR
AND

REAL EXAMPLE:

gf3 c"""
gf3 gs3 gf4 gs3
5 gf3 r gs+ gf4 gs+ w
"""the cat was
gf0 r gf7 w gf0 gs3 gf6 w
OR
...
Oooo I love new languages, your new language looks like https://en.wikipedia.org/wiki/Brainfuck


I want to create a new language too, a fifth generation language:
1st gen: Punch card
2nd gen: Assembly
3rd gen: High level, eg. C
4th gen: High level with OOP
5th gen: High level with OOP and built-in UI markup
Title: Re: Possibly a breakthrough programming language?
Post by: LOCKSUIT on February 01, 2020, 08:00:04 PM
No, brainfuck is said to be designed to be a braincuckle, mine is not. Mine is so simple, trust me. It currently needs some math but all works I think. Just follow the goto and read/write I made in opening post. gf3 means go to line 4....gs5 means go to letter 6 in the line of code you went to WHILE follow the top most line of code not finished yet !

For the video below, compare it to the video above I made of my Blocky code. You can see how my pro. language can manipulate letters to do what I show yous I did in Blocky.

note i messed up at 1:30 i meant the """the etc we already have lol
https://www.youtube.com/watch?v=ZZ8uYfgWeWo
Title: Re: Possibly a breakthrough programming language?
Post by: Hopefully Something on February 01, 2020, 08:42:23 PM
Yey! Blocky FTW!
Title: Re: Possibly a breakthrough programming language?
Post by: LOCKSUIT on February 01, 2020, 09:11:25 PM
BZZZZZ BZZZZZ
intaking wisdom flow into brain
"woosh"
"singularity"

https://www.google.com/search?q=blockly+create+list&tbm=isch&sxsrf=ACYBGNSxcH3pZoIeBnuauGhBKhA4VgEbvg:1580591398854&source=lnms&sa=X&ved=0ahUKEwjCh_TmobHnAhWBZs0KHWuNBfgQ_AUICigB&biw=1280&bih=923&dpr=1#imgrc=AIWw-5v1-_DReM&imgdii=suMWYovyyy_lXM

I like how when you click a complex one, Google Images suggests related images as complex or more complex, giving you complex large block programs , cool
Title: Re: Possibly a breakthrough programming language?
Post by: LOCKSUIT on February 01, 2020, 09:17:14 PM
Note if anyone wants to create my programming language, go ahead, I'm currently fixed on Blocky and otherwise can't code.
Title: Re: Possibly a breakthrough programming language?
Post by: LOCKSUIT on February 01, 2020, 09:23:49 PM
my god, it's working

https://www.youtube.com/watch?v=j1WDy1zVjjw
Title: Re: Possibly a breakthrough programming language?
Post by: ivan.moony on February 01, 2020, 09:27:54 PM
I told you it's fun :)
Title: Re: Possibly a breakthrough programming language?
Post by: LOCKSUIT on February 01, 2020, 10:24:59 PM
Blockly=fun
empty IDE=no fun
Title: Re: Possibly a breakthrough programming language?
Post by: Dat D on February 02, 2020, 02:31:08 AM
Blockly=fun
empty IDE=no fun
Oh mamma mia, I tried to do bubble sort with Blockly and now I know how painful it is.  :uglystupid2:
I can type code in text like a few times faster than drag-n-drop those stuff on Blockly.  :D


My bubble sort:
https://blockly-demo.appspot.com/static/demos/code/index.html#bfen2c
Title: Re: Possibly a breakthrough programming language?
Post by: LOCKSUIT on February 02, 2020, 03:19:22 AM
Nice bubble sort :)

But at least I can code. If I don't I'm out a coder bro. That matters. I can code quite fast in Blocky and see no issue at all. I see all commands, sorted in categories too. That is what is helping me. And their clearness visually, which is key.

All I need is to create just a few programs to make AGI. That's all I want and all I seek. It helps me a ton.
Title: Re: Possibly a breakthrough programming language?
Post by: LOCKSUIT on February 02, 2020, 03:35:44 AM
Also finding the group useful:
https://groups.google.com/forum/#!searchin/blockly/limitation/blockly/l22CIk5mrxo/EFIPUsN830oJ



Their view is this:

Make it easy for the beginner
We chose to have the offset of the first element in a list be 1, rather than 0, because 1 is what a naive user would expect.  While this might make the transition to conventional programming languages a little harder, we’re more concerned with helping people discover the joy of computing rather than the details that don’t make sense until you’ve taken several classes.  Follow the Principle of Least Astonishment.  If you have to decide between astonishing a beginner or an experienced computer scientist, astonish the latter.

It is not always obvious what is easiest for the user.  Consider the numeric argument to the Sound component’s vibrate method.  We chose to make it milliseconds, which enables people to use whole numbers (500 milliseconds) rather than fractions (.5).  On the other hand, someone who doesn’t read the manual or doesn’t know what milliseconds are would probably start their experimenting with an argument of 1, which would cause an imperceptible vibration. 

Organize functionality in a way that makes sense to the user
Continuing our discussion of the vibrate method, a physicist might know to look for it in the Sound component, but most users would not.  Unfortunately, I don’t have a better suggestion of where the vibrate functionality should be, short of creating a new component whose name would make people snicker.

An example of good organization is having Camera, ImagePicker, and VideoPlayer all within the Media category.  Their implementations are totally different, but, to the user, it makes sense for them to be together.

Mobile ≮ Desktop
Mobile devices should not be considered to be desktop (or laptop) computers with small displays.  Focus less on functionality that works better on a large screen than a small screen, and focus more on functionality that takes advantage of mobile devices’ unique features, such as portability, connectivity (SMS, BlueTooth, NFC, etc.), sensors (acceleration and location), and recorders (audio, photographs, videos).  This principle suggests it would be better to develop components for data collection, taking advantage of all of these features of mobile devices, than to develop, for example, the capacity to display multiple videos on a single screen, which could only be done poorly on (existing) mobile technology.

Provide default values
Users should not have to understand all of a component’s properties in order to use it.  For example, new Labels have reasonable default values for all of their properties except for Text (which has the self-explanatory initial value “Text for Label1”).  This enables someone to begin using a component quickly and only look at other properties when dissatisfied with the default behavior (such as a label being hard to read on their chosen Screen background image).  By not requiring the user to understand properties until they are needed, this makes them a solution rather than a problem.

Similarly, reasonable default values should be provided for built-in blocks.  The parameter to the “make color” block is a list that must contain elements with the values of the red, green, and blue components and may optionally contain a fourth element with an alpha level, something that most users will never need.  (The downside of taking a single list parameter instead of one parameter for each numeric input is that the socket labels are less meaningful.)
Title: Re: Possibly a breakthrough programming language?
Post by: LOCKSUIT on February 02, 2020, 04:01:26 AM
Submitting data. Read it all if you're interested in AGI.

https://www.youtube.com/watch?v=fo7F_KfVgB8
Title: Re: Possibly a breakthrough programming language?
Post by: LOCKSUIT on February 02, 2020, 04:16:03 AM
This thread keeps getting better, wow, HS will like this CAD:

https://www.blockscad3d.com/editor/

http://play.makerscad.com/

https://blockly-demo.appspot.com/static/demos/index.html
Title: Re: Possibly a breakthrough programming language?
Post by: Hopefully Something on February 02, 2020, 05:25:37 AM
Good on you. Finally we're getting somewhere. I just ordered the necessary chips and download cables to go ahead with a Blockly build.. for my blocks heheh.
Title: Re: Possibly a breakthrough programming language?
Post by: LOCKSUIT on February 02, 2020, 06:16:39 AM
( ͡° ͜ʖ ͡°)( ͠° ͟ʖ ͡°)( ͡~ ͜ʖ ͡°)( ͡ʘ ͜ʖ ͡ʘ)( ͡o ͜ʖ ͡o)(° ͜ʖ °)( ‾ʖ̫‾)( ಠ ͜ʖಠ)( ͡° ʖ̯ ͡°)( ͡ಥ ͜ʖ ͡ಥ)༼ ͡° ͜ʖ ͡° ༽(▀̿Ĺ̯▀̿ ̿)( ✧≖ ͜ʖ≖)(ง ͠° ͟ل͜ ͡°)ง(͡ ͡° ͜ つ ͡͡°)[̲̅$̲̅(̲̅ ͡° ͜ʖ ͡°̲̅)̲̅$̲̅](✿❦ ͜ʖ ❦)ᕦ( ͡° ͜ʖ ͡°)ᕤ( ͡° ͜ʖ ͡°)╭∩╮¯\_( ͡° ͜ʖ ͡°)_/¯(╯ ͠° ͟ʖ ͡°)╯┻━┻( ͡°( ͡° ͜ʖ( ͡° ͜ʖ ͡°)ʖ ͡°) ͡°)¯\_(ツ)_/¯



٩(^ᴗ^)۶ for FREE
Title: Re: Possibly a breakthrough programming language?
Post by: Freddy on February 02, 2020, 10:32:13 AM
It would be great if Lock managed to cause the singularity in Blockly  ;) :)
Title: Re: Possibly a breakthrough programming language?
Post by: LOCKSUIT on February 02, 2020, 06:53:56 PM
I'm going to see if I can today in Blockly.

Pro Tip: Even if you can't make your full algorithm in Blockly, it still Generates the code you did make and all you need to do is ask Google/Upwork data science programmers to fill in those missing lines of code. Furthermore I can easily read the code now using blocks, and the python etc code too, comparing the blocks to the code, meaning I can learn programming easier too! Lastly I can hire a C++ coder to make identicle algorithm else no payment - this works if have a exact result and I did it once for 400USD.

Currently I'm looking for a timer and how to glue 2 substrings together, in Blockly, hmm. @Dat D, I think there is a list block that sorts items in ascending or descending order, lol.

EDIT
https://groups.google.com/forum/?_escaped_fragment_=topic/blockly/KFEMJ_EkZuw

-see below for the joing strings sorry-

Looks like all 3 are solved, no issues then!
Title: Re: Possibly a breakthrough programming language?
Post by: LOCKSUIT on February 02, 2020, 09:24:53 PM
This is not funny though:

from
https://developers.google.com/blockly/guides/app-integration/best-practices
"9. Exit strategy
Block-based programming is often a starting point for programming. In the context of teaching computer programming, it is a gateway drug that gets students addicted, before moving them on to harder things. How long this block-based programming period should last for students is hotly debated, but if your goal is to teach programming it should be temporary."
Title: Re: Possibly a breakthrough programming language?
Post by: LOCKSUIT on February 02, 2020, 10:01:04 PM
Apparently to save your project u copy n paste the XML to ur computer, omg. OK.

And download the .zip and go to code folder to get the desktop webpage tab working environment. Then paste your XML into there to LOAD your project.

Too bad they were smart enough to make it but careless enough to finish it how they should, the docs suck.

But it looks good enough to grant my wishes.
Title: Re: Possibly a breakthrough programming language?
Post by: LOCKSUIT on February 02, 2020, 10:11:47 PM
the joining the substrings, paste this into the XML area:

<xml xmlns="https://developers.google.com/blockly/xml">
  <block type="text_print" id="x:t#ZwiUc(@@^TljtR1?" x="212" y="62">
    <value name="TEXT">
      <shadow type="text" id="MjKvQYI;.2AE!vRBsdN/">
        <field name="TEXT">abc</field>
      </shadow>
      <block type="text_join" id="(+w,PFmQ;{dId2h1/iWy">
        <mutation items="2"/>
        <value name="ADD0">
          <block type="text" id="O^:r9}myRNtH3#@*7V.1">
            <field name="TEXT">hi</field>
          </block>
        </value>
        <value name="ADD1">
          <block type="text" id="p;sP~jn=B*{s/[+vBW6[">
            <field name="TEXT"> there</field>
          </block>
        </value>
      </block>
    </value>
  </block>
</xml>
Title: Re: Possibly a breakthrough programming language?
Post by: Hopefully Something on February 02, 2020, 10:17:30 PM
Pictures are so helpful but get zero respect. Would be nice to take this from a picture book or training wheel perspective, to a textbook diagram or power steering sort of thing.
Title: Re: Possibly a breakthrough programming language?
Post by: LOCKSUIT on February 02, 2020, 10:27:04 PM
Unless I can as--quickly realize the code_2_myBlocks "relation", I should stay with just blocks. I don't want to learn a new language. So unless it's somehow super easy to understand the code it generates, I'll be keeping my eyes on the blocks :)
Title: Re: Possibly a breakthrough programming language?
Post by: LOCKSUIT on February 03, 2020, 04:40:04 AM
So ya the code it generates is readable, small, and I probably can learn Python faster using the blocks. But I'll try to stay with blocks.
Title: Re: Possibly a breakthrough programming language?
Post by: LOCKSUIT on February 03, 2020, 06:06:19 PM
I'm happy I can use Blocky with no training as I made a few algorithms in it already. Versus a year to learn Python, that's a relief it's gone. I did 3 years ago read the manual for EV3 Lego Mindstorm's block idea and was easy enough to use it instantly as well, which is simple to grasp, so possibly my brain is simply easily recognizing the variables, loops, etc. So I recommend anyone to read EV3 manual if Blockly looks confusing. I do have 4 years of experience studying AGI, but I did stay away from programming 93% of the time. Though what I study/build is a Theory Of All, intelligence, physics, evolution, so that could certainly be helping me in any domain remotely related lol. It doesn't make me perfect though, I'm just human.
Title: Re: Possibly a breakthrough programming language?
Post by: ivan.moony on February 03, 2020, 07:29:13 PM
See: railroad diagram (https://en.wikipedia.org/wiki/Syntax_diagram)
Title: Re: Possibly a breakthrough programming language?
Post by: LOCKSUIT on February 04, 2020, 01:28:42 AM
The only other as good PL I found I thinkkkkk is this one, tryyyying to see it but their vids are scarce and crappy. And it costs 500$ a year so why even bother right. Ok back to coding in Blockly. Blockly probably has all we need and looks prettier too.

PDF:
http://www.ni.com/pdf/manuals/321296b.pdf

https://www.youtube.com/watch?v=SPraUAM1KvQ
Title: Re: Possibly a breakthrough programming language?
Post by: LOCKSUIT on February 04, 2020, 01:40:11 AM
oops, wrong PDF, this one has the many many blocks:
Title: Re: Possibly a breakthrough programming language?
Post by: LOCKSUIT on February 04, 2020, 09:25:12 PM
I easily solved the remaining issues I came to. The open/write to desktop, the convert float to string to int. The bits to bytes or vice versa I didn't solve yet but should be easy.

solutions for the 2 above:

a = None
b = None
a = 0.0036474577
b = str(a)[2 : 6]
print(b)

with open('input.txt','r',encoding='utf-8') as f:
    text = f.read()

just change your variable in other places to open to ex. text if you have text = f.read() as in the above
Title: Re: Possibly a breakthrough programming language?
Post by: LOCKSUIT on February 04, 2020, 10:23:54 PM
Lol the Google creator PMed me from the group for Blockly saying:


"Please stop sending emails to the list before you have investigated your issues.  You are spamming a lot of people with these updates.

If you keep sending messages at this frequency I will have to lock the thread.

Rachel"


After I said

"Another update: Perhaps another issue solved, easily again. What gives? So easy but maybe I'm messing up the system Blockly follows? I pasted Blockly's Python code it generated into PyCharm and I changed the line "a = b[2 : 6]" to "a = str(b)[2 : 6]" so that it converts float to string finally:

a = None
b = None
a = 0.0036474577
b = str(a)[2 : 6]
print(b)

it outputs as I desired: 0036

It'd seem to me that adding that str() wouldn't harm anything else in the code. I just don't understand why Blockly after 9 years of development didn't make a solution for this. Maybe the 5 languages all don't allow type conversion. But they could add a switch for that to unlock these few missing blocks."
Title: Re: Possibly a breakthrough programming language?
Post by: LOCKSUIT on February 04, 2020, 10:30:49 PM
Their poor docs and browser based system is the cause of my questions. They're just money hungry.
Title: Re: Possibly a breakthrough programming language?
Post by: LOCKSUIT on February 06, 2020, 04:26:10 AM
So far I did a lot of looking around and Blockly is really really good and very simply/free. The only workarounds needed so far are:


read/writes files ---- i can use the code below, and to open your block code paste/save the XML code btw.
split by each char ---- only 3 times slower if use blocks i.e. 40 seconds for 100MB and can use the code above if want too.
string<>integer<>float conversion ---- use code below
bits to bytes ---- I can do manually or by other means.
-------------------------------------------------------------------------
with open('input.txt','r',encoding='utf-8') as f:
    text = f.read()
//just change your variable in other places to open to ex. text if you have text = f.read() as in the above

def split(text):
    return list(text)
print(split(text))

a = None
b = None
a = 0.0036474577
b = str(a)[2 : 6]
print(b)
Title: Re: Possibly a breakthrough programming language?
Post by: LOCKSUIT on February 07, 2020, 09:04:07 PM
Ok I looked at most pages (just not the non critical ones) in https://www.learncpp.com/ and it seems Blockly has everything you need to code, easily. Assuming you follow the below (I added 2 lines at bottom). PLs also seem more simpler than I thought.



with open('input.txt','r',encoding='utf-8') as f:
    text = f.read()
//just change your variable in other places to open to ex. text if you have text = f.read() as in the above

def split(text):
    return list(text)
print(split(text))

a = None
b = None
a = 0.0036474577
b = str(a)[2 : 6]
print(b)

So:
read/writes files ---- i can use the code above, and to open your block code paste/save the XML code btw.
split by each char ---- only 3 times slower if use blocks i.e. 40 seconds for 100MB and can use the code above if want too.
string<>integer<>float conversion ---- use code above
bits to bytes ---- I can do manually or by other means.
+
lets make sure Blockly's Python doesn't pass (copy) a list when use it!.....use Call Function holding the array if can't
and apparently using a reference saves on memory i.e. don't assign a long array or variable to x or copy it, just do so to a pointer representing it
Title: Re: Possibly a breakthrough programming language?
Post by: LOCKSUIT on February 08, 2020, 07:35:45 AM
almost done!!
https://blockly-demo.appspot.com/static/demos/code/index.html#mzd28f
And I didn't even refactor yet.
how u like my list-tree?
Title: Re: Possibly a breakthrough programming language?
Post by: LOCKSUIT on February 10, 2020, 07:54:36 PM
Had to restart. My code wasn't readable. Is this tree below small enough code? It takes 3 mins on 10MB input at 17-letter tree branches but uses 13GB RAM.

edited link sorry
https://blockly-demo.appspot.com/static/demos/code/index.html#f6wbqn

code is below after removing useless code

tree = ['']
window_start = 1
window_end = 4
for count2 in range(25):
  window = 'abbacbccacbbbababcabccacabcbcb'[window_start - 1 : window_end]
  window_start = window_start + 1
  window_end = window_end + 1
  char_location = 1
  node = 1
  test = 1
  for count in range(4):
    char_in_window = window[char_location - 1]
    char_location = char_location + 1
    print(tree)
    if tree[node - 1].find(char_in_window) + 1 == 0:
      tree[node - 1] = str(tree[node - 1]) + str(char_in_window)
      if char_location != 5:
        if node == len(tree):
          tree.append([])
        tree[(node + 1 - 1)].append(len(tree) + 1)
        tree.append('')
      node = len(tree)
    else:
      if test == 4:
        goto = node + 0
      else:
        goto = node + 1
      node = tree[node - 1].find(char_in_window) + 1
      node = tree[goto - 1][node - 1]
      test = test + 1
Title: Re: Possibly a breakthrough programming language?
Post by: LOCKSUIT on February 10, 2020, 10:00:37 PM
i refactored, code is smaller:
https://blockly-demo.appspot.com/static/demos/code/index.html#u5tpn3
Title: Re: Possibly a breakthrough programming language?
Post by: Hopefully Something on February 11, 2020, 01:24:01 AM
Wow that's a lot of progress. I'm still stuck on the first question.

(https://i.ibb.co/6BVd1Gf/Screenshot-772.png) (https://ibb.co/nCZWn45)
Title: Re: Possibly a breakthrough programming language?
Post by: LOCKSUIT on February 11, 2020, 02:02:49 AM
New refactor; not smaller code but less blocks!

New:
https://blockly-demo.appspot.com/static/demos/code/index.html#axnwxx

Previous:
https://blockly-demo.appspot.com/static/demos/code/index.html#u5tpn3


EDIT:
NEW NEW:
https://blockly-demo.appspot.com/static/demos/code/index.html#tzh87z
Title: Re: Possibly a breakthrough programming language?
Post by: LOCKSUIT on February 11, 2020, 06:00:56 AM
I smell a refactor coming on!:
https://blockly-demo.appspot.com/static/demos/code/index.html#eronkg
Title: Re: Possibly a breakthrough programming language?
Post by: LOCKSUIT on February 11, 2020, 06:08:26 AM
Hmm, it has less bvlocks but longer code :)

above version:

from numbers import Number
tree = ['']
window_start = 1
window_end = 4
for count2 in range(25):
  window = 'abbacbccacbbbababcabccacabcbcb'[window_start - 1 : window_end]
  window_start = (window_start if isinstance(window_start, Number) else 0) + 1
  window_end = (window_end if isinstance(window_end, Number) else 0) + 1
  char_location = 1
  node = 1
  for count in range(4):
    char_in_window = window[char_location - 1]
    char_location = (char_location if isinstance(char_location, Number) else 0) + 1
    char_index = tree[node - 1].find(char_in_window) + 1
    if char_index == 0:
      tree[node - 1] = str(tree[node - 1]) + str(char_in_window)
      if char_location != 5:
        if node == len(tree):
          tree.append([])
        tree[(node + 1 - 1)].append(len(tree) + 1)
        tree.append('')
      node = len(tree)
    else:
      if char_location != 5:
        goto = node + 1
      else:
        goto = node
      node = tree[goto - 1][char_index - 1]
print(tree)

previous version:

tree = ['']
window_start = 1
window_end = 4
for count2 in range(25):
  window = 'abbacbccacbbbababcabccacabcbcb'[window_start - 1 : window_end]
  window_start = window_start + 1
  window_end = window_end + 1
  char_location = 1
  node = 1
  for count in range(4):
    char_in_window = window[char_location - 1]
    char_location = char_location + 1
    char_index = tree[node - 1].find(char_in_window) + 1
    if char_index == 0:
      tree[node - 1] = str(tree[node - 1]) + str(char_in_window)
      if char_location != 5:
        if node == len(tree):
          tree.append([])
        tree[(node + 1 - 1)].append(len(tree) + 1)
        tree.append('')
      node = len(tree)
    else:
      if char_location != 5:
        goto = node + 1
      else:
        goto = node
      node = tree[goto - 1][char_index - 1]
print(tree)
Title: Re: Possibly a breakthrough programming language?
Post by: LOCKSUIT on February 11, 2020, 07:00:54 AM
now this is getting interesting, conversion type blocks and more? Code2Blocks? Hmm....sweet!
My code makes it generate the blocks, paste my code from above :)
Seems buggy though, I like Google's.
https://think.cs.vt.edu/blockpy/blockpy/load?settings-display.instructor=true
Title: Re: Possibly a breakthrough programming language?
Post by: LOCKSUIT on February 11, 2020, 07:26:21 AM
Wow, 45 seconds for the bigger code (post # 48), 30 secs for the smaller code. So it's a keeper hmm....
Title: Re: Possibly a breakthrough programming language?
Post by: LOCKSUIT on February 11, 2020, 07:52:42 PM
refactor!
https://blockly-demo.appspot.com/static/demos/code/index.html#xhyzuf

tree = ['']
window_start = 1
window_end = 4
for count2 in range(8000000):
  window = text[window_start - 1 : window_end]
  window_start = window_start + 1
  window_end = window_end + 1
  char_location = 1
  node = 1
  for count in range(4):
    char_in_window = window[char_location - 1]
    char_location = char_location + 1
    char_index = tree[node - 1].find(char_in_window) + 1
    if char_index == 0:
      tree[node - 1] = str(tree[node - 1]) + str(char_in_window)
      if node == len(tree):
        tree.append([])
      tree[(node + 1 - 1)].append(len(tree) + 1)
      tree.append('')
      node = len(tree)
    else:
      if char_location != 5:
        goto = node + 1
      else:
        goto = node
      node = tree[goto - 1][char_index - 1]
print(tree)
Title: Re: Possibly a breakthrough programming language?
Post by: LOCKSUIT on February 11, 2020, 08:30:27 PM
REFACTORRRRRRRRRR!!!!!!!!!!!!!!!!!!
https://blockly-demo.appspot.com/static/demos/code/index.html#95fn2y

tree = ['']
window_start = 1
window_end = 4
for count2 in range(2):
  window = 'ababa'[window_start - 1 : window_end]
  window_start = window_start + 1
  window_end = window_end + 1
  char_location = 1
  node = 1
  for count in range(4):
    char_in_window = window[char_location - 1]
    char_location = char_location + 1
    char_index = tree[node - 1].find(char_in_window) + 1
    if char_index == 0:
      tree[node - 1] = str(tree[node - 1]) + str(char_in_window)
      if node == len(tree):
        tree.append([])
      tree[(node + 1 - 1)].append(len(tree) + 1)
      tree.append('')
      node = len(tree)
    else:
      node = tree[(node + 1 - 1)][char_index - 1]
print(tree)
Title: Re: Possibly a breakthrough programming language?
Post by: LOCKSUIT on February 11, 2020, 09:26:56 PM
RE-FACT-OR

tree = ['']
window_start = 1
window_end = 4
for count2 in range(2):
  window = 'ababa'[window_start - 1 : window_end]
  window_start = window_start + 1
  window_end = window_end + 1
  char_location = 1
  node = 1
  for count in range(4):
    char_in_window = window[char_location - 1]
    char_location = char_location + 1
    char_index = tree[node - 1].find(char_in_window) + 1
    if char_index == 0:
      tree[node - 1] = str(tree[node - 1]) + str(char_in_window)
      if node == len(tree):
        tree.append([])
      tree[node].append(len(tree) + 1)
      tree.append('')
      node = len(tree)
    else:
      node = tree[node][char_index - 1]
print(tree)
Title: Re: Possibly a breakthrough programming language?
Post by: LOCKSUIT on February 11, 2020, 09:42:24 PM
RE FACT OR

e = ['']
h = 1
g = 4
for count2 in range(2):
  f = 'ababa'[h - 1 : g]
  h = h + 1
  g = g + 1
  c = 1
  d = 1
  for count in range(4):
    a = f[c - 1]
    c = c + 1
    b = e[d - 1].find(a) + 1
    if b == 0:
      e[d - 1] = str(e[d - 1]) + str(a)
      if d == len(e):
        e.append([])
      e[d].append(len(e) + 1)
      e.append('')
      d = len(e)
    else:
      d = e[d][b - 1]
print(e)
Title: Re: Possibly a breakthrough programming language?
Post by: LOCKSUIT on February 11, 2020, 09:45:22 PM
R E F A C T O R

e = ['']
g = 4
for count2 in range(2):
  f = 'ababa'[count2 : g]
  g = g + 1
  c = 1
  d = 1
  for count in range(4):
    a = f[count - 1]
    b = e[d - 1].find(a) + 1
    if b == 0:
      e[d - 1] = str(e[d - 1]) + str(a)
      if d == len(e):
        e.append([])
      e[d].append(len(e) + 1)
      e.append('')
      d = len(e)
    else:
      d = e[d][b - 1]
print(e)
Title: Re: Possibly a breakthrough programming language?
Post by: LOCKSUIT on February 11, 2020, 09:49:36 PM
ROTCAFER

e=['']
g=4
for count2 in range(2):
  f='ababa'[count2:g]
  g=g+1
  c=1
  d=1
  for count in range(4):
    a=f[count-1]
    b=e[d-1].find(a)+1
    if b==0:
      e[d-1]=str(e[d-1])+str(a)
      if d==len(e):
        e.append([])
      e[d].append(len(e)+1)
      e.append('')
      d=len(e)
    else:
      d=e[d][b-1]
print(e)
Title: Re: Possibly a breakthrough programming language?
Post by: LOCKSUIT on February 21, 2020, 11:29:09 PM
By today I SHOULD be done the AI compressor. Just fine tuning it now...
Title: Re: Possibly a breakthrough programming language?
Post by: LOCKSUIT on February 23, 2020, 09:16:37 AM
it seems my alg without its enhancing function already compresses

"thezcatzatzthezczazcatzthatzazcatzthezcatzthezcat"

into

0636763912451316

which in bits is is 10010000110010001000100110110011101011010011110100

so 50 bits to store 392 bits
Title: Re: Possibly a breakthrough programming language?
Post by: LOCKSUIT on February 23, 2020, 09:25:28 AM
see it wants patterns :)

for "azctheztaehheczathaeezzccaahazcezeccaaathtaczaeta"
959365642149805036228786047993219

for "thethethecatthecatthththaccattheacatthethethethet"
9611206569478968

for "bbbaaabbbaaabbbaaabbbaaabbbaaabbbaaabbbaaabbbaaab"
199905
Title: Re: Possibly a breakthrough programming language?
Post by: LOCKSUIT on February 24, 2020, 06:53:59 AM
it's working! Almost. Decoder works but then screws it up lol.
Title: Re: Possibly a breakthrough programming language?
Post by: LOCKSUIT on February 24, 2020, 07:48:47 AM
it worked, it decompressed

now time to upsize it for real etc
Title: Re: Possibly a breakthrough programming language?
Post by: LOCKSUIT on February 25, 2020, 12:17:57 AM
Look at that! My less that 100 lines of code - with just this text and a window of 4 letters!


input was 6,224 bits;

i was at the school and was very happy that my new dog would be there and yes i love dogs and cats so much so i knew he was going to be loved by others as well and so i kept walking around the school and my cat also was there at the school and the dog seen the cat chase the others and was so happy that we could all be there yes i was very happy to see my cat and dogs all there with my friends and so we wanted to do something special and the best thing we could do was go to the store near the school and the cats were there ready to buy them and the seller was happy to sell them also so we baught some more cats indeed yes we did and brought them to the school with our other cats and dogs where we could party more and see each other have more fun with the cats and the sc


output 2,293 bits;

1000100000110110011101011101100000100111000101000100110001000110110000111100011011001001000101000001011111000010010101010110000111011010001100100111000101010110001111100000110110111100111010101000110110110001010110111111100110111100100110110101111000110110111111110111101000000110000010011001110011110000100011011001001001101111011011111011011000011110011101001101100100110011000010001110110001110100101100111111101100101010000100000100101110000001011100000110110111010101110101001111101001011100111001000010111111101100001101110010001000110111111100011110011011001010010101010001111111001100111110101010010000000110101111010000100011000101000100000011110100110110110000011111001011000110010010101001101010011000101111010000010100010110101000110100011010101100000010001010101111100010111011101110111001011110010100100001011101100111010111100010011000100011010101000100000101111001011100010101100111000000101000010100100000001011011011111001100101110110011100111011110010100111111111000100011101111101111100011010010111111101110001110010000000000110000110001101101111110100110001001100100001111000111011011011000010111111001111001100110000111000110111111011010011101010100111101110110011101110000001100100101011000111101011111011001000111001110001101000010101011001010001000001000011110101001100010011101100001011001001101011000001100000011010011100000111110100010001011110010010001010111100001111101101000011011101111000001010001101110011111101001010010101110111100101010001101101011001110011110101010100100001101001000111111001000011001001000110011000111011000011110110001110010101000110000101110111100000011100100000010101101000101100111000011011101001110000100111110111101000111111001010010101001001110011111101100110010011101010001011110011011111010011111110111000000011000101000010011110011100001111110100000011000100000001000001000101100111010110000010011001000010011011110000101001110001111110111101001101101101110101000110000110100000111011010000110110110010000110010011001110010100011100100100100011111110000110011001000111011001101001101010110101110100011101000100111100000011100011010111001100101100010110001001101000011111100111110001000001001000001000110110101000100101100111101101111101101001110001100101001010110001011010111000101100111111111101100000000110110000111101000001100

Decompressor made it back regenerated perfectly.
Title: Re: Possibly a breakthrough programming language?
Post by: WriterOfMinds on February 25, 2020, 01:20:34 AM
That may not be a numerically impressive compression percentage yet, but I'm glad you coded your own algorithm.  Good job.
Title: Re: Possibly a breakthrough programming language?
Post by: LOCKSUIT on February 25, 2020, 08:49:48 AM
So far 266,731 bytes input - compressed approx. 77,000 bytes.....3.4x compression. I didn't finish the mixing algorithm yet =)

will show code shortly

I made the code fully in Blockly, from scratch, tree and all, in just under 100 lines of code

it does Online Learning, Arithmetic Coding, context mixing...
Title: Re: Possibly a breakthrough programming language?
Post by: Freddy on February 25, 2020, 09:35:00 AM
Well done Locksuit - so you are enjoying the programming at last then?  8)
Title: Re: Possibly a breakthrough programming language?
Post by: LOCKSUIT on February 25, 2020, 10:12:44 AM
Yeah, it's super easy plus fun now with Blockly, I am a pro programmer now.
Title: Re: Possibly a breakthrough programming language?
Post by: Art on February 25, 2020, 02:40:23 PM
Yep...Bits...Bytes...what's the difference... O0
Title: Re: Possibly a breakthrough programming language?
Post by: LOCKSUIT on February 27, 2020, 09:20:12 AM
Ok hot shots, time to release my open source code. I will probably have to limit my future releases to follow modern safety practices and profit (openAI joke). I made this from scratch in Blockly, tree, Online Learning, searching, context mixing, arithmetic encoding, decompressor. It can compress 100MB down to ~23MB. I skipped refactoring it for now, it is still small code too.

https://blockly-demo.appspot.com/static/demos/code/index.html#

Attached is the big input version, try that one, the Blockly one is just to show the toy version. It makes 266,700 bytes into 74,959 bytes and someone's compressor that makes 100MB into 21.8MB makes the 266,700 bytes into 70,069 bytes. To switch to decompression put 'no' at the top and make the input only have the first 15 letters, not all the input of 266700 letters, and put the code it encoded into the decode input at top ex. 0.[here]. You can use the following link to make the code into bits https://www.rapidtables.com/convert/number/binary-to-decimal.html (although I just divide the length of the encoding by 3, then make 1 3rd * 4, and 2 3rds * 3, which results in approx. same bit length. Ex. you create 0.487454848 which is 9 digits long, so: 9 / 3 = 3, 3*4=12 and 6*3=18, 18+12=30, 0.487454848  = 30 bits long!
Title: Re: Possibly a breakthrough programming language?
Post by: LOCKSUIT on February 29, 2020, 11:14:20 PM
in case the link didn't show the blocks, here. Did it?
Title: Re: Possibly a breakthrough programming language?
Post by: LOCKSUIT on March 01, 2020, 04:29:34 AM
Btw that XML goes into Blockly. At top of Blockly is the XML tab.
Title: Re: Possibly a breakthrough programming language?
Post by: LOCKSUIT on March 07, 2020, 10:51:29 AM
cross posting this:

I personally use lossless compression as a true evaluation for finding AGI, which is the goal behind the Hutter Prize. Of course it doesn't cover everything, AI minds need bodies/nanobots, rewards, surf the web, etc. Yet, so far the best compressors talk to themselves and do Online Learning and rearrange articles more neatly and group similar words, etc. So it seems like a million dollar idea. Makes you wonder is there any other evaluation we need if my goal is to massively lower the probability of death/pain on Earth? (That's where Earth is going with evolution and the Hutter Prize, it repairs missing data using prediction to make Earth a neater pattern fractal so save on energy wasted, adds it using Online Learning, then it uses its larger context tree to add new data even faster, exponentially growing in scale.) If I invent true AGI, it uses higher intelligent means (faster than brute force) to make/predict the future and form the settled state ('utopia') that physics will settle us down into, and so does it need to have a body (do we need another evaluation) if it knows enough about the real world that it can extract any unseen text/image it would have seen by a body? It has less need to gather real data. So in a sense, to invent AGI is to just predict well and smash the Hutter Prize. As for the rest of the evaluation for achieving immortality, we do need bodies to carry out tasks still, intelligence doesn't 'do it all', but we already have humanoid bodies with no brain really (and humans etc), so yes all we need to focus on is AGI. And big data. Yes the other evaluation being part of the intelligence evaluation is data/brain size; scale. Scale and Prediction is what we need to work on only. Note your prediction can be good but slow/large RAM needed, but it is less important and we can feel that effect i.e. it tells us the cure for cancer but had to wait a year = a big thank you still. As with scale, the more workers on Earth also the more faster we advance as well.

The first thing you notice is it is easy to get the enwiki8 100MB down to 25MB, but exponentially harder the lower you go. oh. So is intelligence solved? No. But you saw people are getting the data compression needs they need already and won't benefit much more now, so it seems! If a company compresses DNA data down from 100MB to 15MB, why would they/we work on the Hutter Prize so much to get 13MB? Here's why: What if they had not 100MB, but 100GB? the amount cut off is now not 2MB, but 2GB! Also, the more data fed in, the more patterns and the more it can compress it! So not 2GB, but 4GB; 100GB becomes 11GB. Mind you, it is funny our AGI prize is being used to compress data as a tool. Though AI can think up any invention itself, so it doesn't seem odd exactly. Now, seeing that we get more compression ratio the more data fed into it, this means if we make the AI predictor more intelligent in finding/creating patterns, it Will result in a huge improvement, not 15MB>14MB>13.5MB>13.45MB>13.44MB. However I'm unsure that makes sense, maybe it is indeed harder the lower you go if we look at a ex. 100TB input ex. the limit is 2TB (and currently we'd ex. only reach 4TB). Perhaps this is harder the lower you go because there is less hints and higher uncertainty in some problems that require lots of knowledge. So instead it is more honoring to lower compression by just a little bit once it is lower. Of course probability gets worse for hard problems to predict well on, consider the question below for predicting the likely answer:

"Those witches who were spotted on the house left in a hurry to see the monk in the cave near the canyon and there was the pot of gold they left and when they returned back they knew where to go if they wanted it back. They knew the keeper now owned it and if they waited too long then he would forever own it for now on."
> Who owns what?
> Possible Answers: Witches own monk/witches own canyon/monk owns gold/monk owns house/monk owns cave/cave owns pot/there was pot/he owns it

You can't help that hard issues have less probability of being predicted. But you can improve it still. These issues are hard because they haven't been seen much, no match. Yes, simply entailment and translation is used to see the frequency of the answer! How do I know this ;-; :D? Think about it, if real world data humans wrote says the turtle sat in water and yelped, you can be sure it will yelp, you can be sure a rolling ball will fall of a table, a molecule x will twist like x, etc, and if the words are unseen but similar ex. 'the cat ate the ?' and you seen lots 'this dog digested our bread' then you know the probability of what follows (and what is what, cat=dog using contexts seen prior). This works for rearranged words and letters etc, typos, etc, missing or added words... So simply doing prediction will discover the true answers with highest probabilities. Of course, it may not be exactly as simple as a match is all that's needed and then you get the entailing answer (or translation) ex. 'the cure to caner is '. At least it doesn't seem like it, but it is. This requires you to take the word cancer, look into what entails IT; pain, rashes, what those are as well similar to, what entails THOSE, repeat, commonsense reasoning like BERT translation here yes, giving you more virtual generated data, and so you are finding yes the entailment to 'the cure to caner is ' except the prediction is not directly tied to those very words if you know what I mean.

Now if our predictor was a image predictor or movie generator, and also used text context (multi-sensory context predictor), we would have AGI 'talking' to itself learning new data online. And can share its thoughts of vision to talk to us using vision too. We also use Byte Pair Encoded segmentation like 'sounds' (words...) to activate certain images, it is more efficient, images aren't as abstract in that way, but can be much more detailed low level! We will need both later to get better prediction. And yes, images are words, words describe image objects and they are inversely the same thing.
Title: Re: Possibly a breakthrough programming language?
Post by: LOCKSUIT on March 13, 2020, 02:00:59 PM
last compressor above - 266,700 bytes into 74,959 bytes
new slightly better compressor - 266,700 bytes into 72,584 bytes
Title: Re: Possibly a breakthrough programming language?
Post by: LOCKSUIT on March 16, 2020, 09:04:41 AM
input > 100,000 enwiki8 bytes
Green> 31,518
mine > 31,447

input 200,000
Green > 60,321
mine > 60,394

going to get it low yet, just u wait

startin to write in python editor now, getting more natural lol....saves time heh
Title: Re: Possibly a breakthrough programming language?
Post by: LOCKSUIT on March 17, 2020, 08:30:44 AM
Green keep in mind go the 100MB down to 21.8MB

20,000 enwiki8 bytes input:
green: 6,849
my last: 6,829
my new: 6,811

100,000 enwiki8 bytes input:
Green: 31,518
my last: 31,363
my new: 31,266

200,000 bytes
green > 60,321
mine > 60,057
Title: Re: Possibly a breakthrough programming language?
Post by: LOCKSUIT on March 19, 2020, 10:27:52 AM
new:

20,000
6,737

200,000
59,457
Title: Re: Possibly a breakthrough programming language?
Post by: krayvonk on March 19, 2020, 01:59:27 PM
Are those actual compression results or just speculations?

Heres some speculations from me to u ->
If it was somehow true that you developed an environmental model off a huge amount of text information, via copying it,  it actually hasnt generated anything new from it, but from my theory it is exactly what was put into it.  (just like the world is exactly what was put into us, if...  we were just copying it.)

You have to read this paper ->  https://worldmodels.github.io/  (thats actually the graphics generating out of its system running in web java, its the best thing on the internet for me.)  This goes a step further than just copying I think,  and its actually converting the information into something else.

If you want to do that,  you need to have another policy, other than just "copy the data."  Which is a policy of "I must be 100% truthful to it."  Thats what this compression system is doing.   Do you remember Marcus Hutter talk about Ockams razor?  This is another policy of "I must be the MAXIMUM compression"  and this is another attempt at trying to form the data, in a form where it becomes a procedural generation of the data, instead of a direct frame spitout. (which logically we think must be larger, than the procedural generation.)
If your doing the compression without that in mind,  your doing back propagation instead of evolution search or gradient descent, it seems the same, but I dont think it is as creative a learning system!


Im not doing that anymore,  I think that if you need another kind of policy, and I think it needs to be a goal based system, where the robot wants to do something inside the world, and the idea goes,  the model doesnt just copy, and it transmutes it to a form which is more useful to the robot, AS it is COPYING, to stay 100% truthful to the laws and behaviour around it.

Theres a big problem already stumping me, and its the fact this goal code, has to B hand written interpretable stuff, where it has to involve only basic word and symbol type knowledge because thats all we have detectors for!  Theres no such thing as a lie detector,  theres no such thing as a "hurt someones feelings" detector, without being a quite shallow and bloated version hit and missy of what it actually is to have a real detector for it, which is impossible to write.

But think,  if it grew some kind of facility around this primitive goal which transmuted the data into instrumental goals and variable and symbol conversions you kinda fantasize about and it really worked like magic and could actually communicate and act,  if you asked it to "please go cure cancer for us?" Then how are u supposed to drive the robot to do it - with only basic detectors, if the whole shabbam worked it would be hit and miss if it even felt like doing it.  ITS ONLY IF THE ROBOT FELT LIKE IT!!!   so big probs there.

As in, you used some certain clever policies,  but what the robot finishes up at YOUVE GOT NO IDEA ABOUT!!!
Title: Re: Possibly a breakthrough programming language?
Post by: LOCKSUIT on March 19, 2020, 03:05:54 PM
Those are actual test results, and decompression worked - exact file match. I used 200,000 bytes from the enwiki8 dataset used in the Hutter Prize. They use enwiki9 now as of this year.

My algorithm is a text generator, a letter predictor using probabilities, learnt Online as it talks to itself storing what it says to itself. It mixes 13 context models and then does Arithmetic Coding. It uses the past text to predict/generate the future until it reaches the end of the file. My algorithm builds a tree from scratch and during mixing will use global model weight, weight threshold squashing, and adaptive threshold based on how many channels are open to it. If I stop its losslessness it generates mostly silly ramblings that were not in the dataset but somewhat make sense. But I will get it as good as the great GPT-2 yet. This works because my algorithm handles future uncertainty with missing data it didn't see yet aka no very likely answer, so it mixes models to get more virtual data to handle uncertainty. My tree uses frequency for phrases to single letter models. And I haven't even started, there's energy, translation, etc etc to do yet that others have paved the way in already and I'm building on it soon. I'm on the way to building REAL AGI here, I'm telling you this is the way to get started. The models are in the tree, it stores text and frequency and can store semantics in the future, the tree is the neural network basically and that's where the pruning and storing of good enough nodes will result in a robust distributed net that is small, fast, and can recognize long unseen strings.

I coded it all from scratch, no reusing others code.

As for the copy while mutate thing in evolution, AGI must answer questions never seen before, i.e. it uses surrounding context atoms to predict/make the future ground/sentence (babies in the womb, text generators...). During disabling lossless compression / Arithmitic Coding, it can generate unseen futures that are likely based on the context past. Sounds like physics. And that mutates the future correctly while clones the sentence topic being on topic and generates content.

As for motivation, we are working on immortality, and AGI which is meant for immortality indirectly! Same for hard drives etc...humans seek food sex for survival and that leads us to homes, hard drives, AI, cars.... Now, some organisms are lazy, but AIs can cheat that easy. So all good.
Title: Re: Possibly a breakthrough programming language?
Post by: krayvonk on March 19, 2020, 03:45:11 PM
Shit!  Coding from scratch now!  CONGRATULATIONS ON GETTING UR SHIT TOGETHER.
Your sounding alot better at it than before,  how long have you been coding for - not long?  I cant believe your kickin butt so much.

You know a couple of terms I dont know ->
      * global model weight
      * weight threshold squashing
      * <ed> virtual data?  - is that generated off the original data,  wouldnt that be illogical?

I gotta watch out when your lingo takes over mine and you steal my bloody job.


But I have some critiques for you ->

- what is a context model - and why have more than 1 of them?  - doesn't it just become 1 in the end? or you do choose from exclusively them as you go?

- using frequency sounds interesting,  but that's running on an illogical policy, thats just "the most common thing is the answer" - and thats a correllation!! - it doesnt matter how many people in bullshit, it doesnt turn it into chocolate cake.

-If your unseen futures are likely based apon the context past,  thats all any intelligent creature can do, and its all im planning on doing AS WELL! but its NOT if it just COPIES the data!   If you want an unseen future that came from its past, thats a step up from just regurgitating it exactly again, I think you have to transfer the semantics from text, to another representation, based apon ANOTHER POLICY! and u have to mutate it AT LEAST.   then it might happen, but my theory could be wrong too... of course.   Im only just coming up with an idea a day.

I coded 2000 lines this week tho bud,  how bout you?

EDIT -> is your system a chat bot?
Title: Re: Possibly a breakthrough programming language?
Post by: LOCKSUIT on March 19, 2020, 04:24:09 PM
I only code from scratch. I don't want others shit in my code!

I started coding when I started using Blockly, you can see at the START of this very thread when i began my rabbit journey !, before than I only never wrote anything but I did try to understand programming language before and used EV3 blocks like 3 years ago so it was easier for me.

      * global model weight
      * weight threshold squashing
      * <ed> virtual data?  - is that generated off the original data,  wouldnt that be illogical?
my alg takes the last 13 windows on previsous text and searches them in the tree......the cat...he cat....e cat....gets next letter canidates.........i mix the 13 sets of predictions together..........a model gets weight based on total count compared to wanted roof to make sure it has enough stats seen............roof adjusts based on total unique symbols being predicted..........total is logistically adjusted so small and large totals are a bit different and smaller than are..........as for virtual data, i just meant the past context activates similar nodes in tree tho mine don't do that yet so much, just the context mixing of models aka prediction sets from the 13 searches, so ya it gets some extra data to help its probability rise correctly, and it adds what it predicts to tree and gets more real data (one day, will get generated data not in dataset), and during prediction it can but not yet can do other tricks to get extra data

mine isnt a chatbot....i explained it a few times now and just above again in lasssst post....but txext generator R a chatbot however!

i can write 10 lines in 10 mins, depends on if is patterny too.....this alg took me 3 weeks to code tho at least and u can see the time by looking at this very thread !....code is about 120 lines, could refactor a lot to make it smaller!

ya my 13 context models can be 1 model in the end yup but for now i take variable search lengths from tree and get children predictions that entail those windows

as for the frequency based on lots of dumb peeps saying it, well, the enwiki8 datset for example is really 'correct' technically, many people r, and u can see the truth inside even if most peeps say something ! .... even i must do this, this is how evolution works! it no has the answers of future yet it gets them from the rubish past it does know.....yup, old toaster becomes fancy toaster all on its own!

lastly i will say it again - in my alg you just shut off the arithmitic coding to not swerve the prediction by taking sometimes not best prediction it thinks and make it pick a top n prediction and it will generate unseen data not in the dataset! And it will be based on high probabilities, sounding like the dataset, but not in it. The predictions i jump into in arithmitic coding to store it lossless are not always highest probability canidates.
Title: Re: Possibly a breakthrough programming language?
Post by: krayvonk on March 19, 2020, 04:44:00 PM
Does it go fast?  I imagine it might only being a 120 line power pocket.
Well done Lock,   Good luck with the A.i. dream, i gotta hit the haywire.
Title: Re: Possibly a breakthrough programming language?
Post by: LOCKSUIT on March 19, 2020, 04:52:01 PM
it's not that fast no, python is slower also........20,000 takes 12 seconds, 200,000 takes x10 longer
but this is my first real program and I'm 24
Title: Re: Possibly a breakthrough programming language?
Post by: krayvonk on March 20, 2020, 08:31:13 AM
Be proud of yourself, cause ur going to be doing this a long time,  and you should print it out frame it and put it on the wall,  then when ppl see they know you can code.
And coding is NOT EASIER OR DISRESPECTED ANY LESS, than doing electronics.  They are both simple,  if you get good at code, your home made computer circuit and nano second custom logic is on the way.   :D
Title: Re: Possibly a breakthrough programming language?
Post by: Korrelan on March 20, 2020, 11:05:02 AM
It pleases me that you are finally coding lock.

 :)
Title: Re: Possibly a breakthrough programming language?
Post by: LOCKSUIT on March 20, 2020, 11:51:30 AM
Didn't even (kinda) need Blockly, yous could have just filmed a video tutorial of python showing me the Flow. It's not that hard to code...
Title: Re: Possibly a breakthrough programming language?
Post by: Art on March 20, 2020, 03:00:54 PM
I'm actually quite proud of you Lock!!  You have come a long way in the past 5 years! Just imagine your future!!  O0
Title: Re: Possibly a breakthrough programming language?
Post by: LOCKSUIT on March 21, 2020, 09:44:23 PM
For the first 1,000,000 bytes of enwiki8
Green gets it to 253,800 bytes
Mine gets it to 251,235 bytes

For the first 10,000,000 bytes of enwiki8
Green gets it to 2,331,508 bytes
Mine gets it to 2,319,337 bytes

Note my parameters can still be tweaked to get that lower a bit.

7zip can't even get it that low lol. ;) Those are ordinary programs.
Title: Re: Possibly a breakthrough programming language?
Post by: LOCKSUIT on March 23, 2020, 01:40:14 PM
new
1,000,000 bytes
251,180 bytes
Title: Re: Possibly a breakthrough programming language?
Post by: LOCKSUIT on March 24, 2020, 01:23:46 PM
So my refined conclusion is Perplexity is worse than Lossless Compression because Lossless Compression forces you to Learn Online, etc, which was amazing for me to code. And the Perplexity test dataset is ok if different actually but it still can be quite similar in some ways in certain areas of the dataset. And LC understands the very data it compresses - Perplexity works on different topic datasets but does lose relatedness if too different which is bad even if only a bit different. And if we want to train on 98% of the internet data, we need test data not duplicated in the data.
Title: Re: Possibly a breakthrough programming language?
Post by: LOCKSUIT on March 31, 2020, 05:17:29 PM
same code, new record (old record is above)
2,308,479 bytes for the first 10,000,000 bytes of enwiki8
Title: Re: Possibly a breakthrough programming language?
Post by: LOCKSUIT on April 01, 2020, 12:57:41 AM
New: Same 10,000,000 bytes losslessly compressed to 2,305,386 bytes. Same code.
Title: Re: Possibly a breakthrough programming language?
Post by: LOCKSUIT on April 04, 2020, 04:50:29 PM
I have a new result for my text compressor (well, predictor).

1,000,000
253,800 - Green
249,682 - Mine

10,000,000
2,331,508 - Green
2,303,936 - Mine
Title: Re: Possibly a breakthrough programming language?
Post by: LOCKSUIT on April 06, 2020, 03:15:17 AM
How my code works:

My algorithm has a 17 letter long window step along the input file 1 letter at a time, updating a tree as it sees new data. The tree's branches are 17 nodes long, and updates node counts if passes any node. For each step the window takes, the algorithm searches the tree for 17 different searches each a letter longer. The children leafs (the final letter of a searched branch) are the predictions with counts seen so far in the file. Layer 1 nodes are children too and need no match. The tree is storing the frequency of all 1/2/3.../17 letters seen so far. The children are what allows you to predict/compress the next letter accurately. These 17 sets of predictions must be mixed because while the longest set is more accurate - we have less statistics, sometimes only 2 counts. We start with the longest found. Ex. 14 letter match in the tree. The 14th set of predictions may say it seen come next a=44, b=33, f=25, w=7. I sum a set's counts up to get a total of (in this case) 109, then I divide each count by the total to get %s that all add up to 1% ex. 0.404% 0.35%.... Now for all these predicted %s, we still have 13 sets to mix and must remove some % from them each. So what I do is I check the total counts of the set against a Wanted Roof ex. 109<>300 (maybe we don't even need to mix lower sets if we got enough stats), and so I cut each % of each prediction by about 1/3rd then in this case. And in this case we still desire 66% more stats. For the next set, if say we have 200<>300, I take away 2/3rds from the 66% - meaning we still desire 22%, not 66% - 2/3rds = 0%! I take away the % got OF the % still desired. A little bit of lower sets always leak in therefore, which is better because we can never be sure even if surpass Roof by lots. Besides, it gave better results. But Roof is decided by how many predicted symbols are in the set (total unique symbols being predicted), so if i have 2 then Roof may be 8 counts wanted. Also, while the Roof is based on how many different symbols are seen in the set, we get a slightly different Roof if we are on the ex. 5th set, i.e. if we have 4 letters in the set #14 then Roof is ex. 33, but if it is set #5 then Roof is ex. 26. Also, based on the Roof's size, a curve's bend is modified. This curve gives small/large total counts in a set an even smaller/larger total (but it isn't used in the Arithmetic Coding, it's only used for deciding how much % this set gets in our mixer). This is meant to be a exponential activation. Finally a global weight is given to each set ex. the 14th set is always given 0.7% of the weight it was going to get lol. I hardcoded the numbers for now but the code isn't grossly large of course. If they were adaptive and were based on the data then the compression would be even better. I just noticed I do exit the mixing before reach lower sets if the Roof is ever surpassed, I'll have to test if this is useful. The Arithmetic Coder takes the combined sets i.e. the prediction %s are combined a, b, c + a, b, c + a, b, c ..... = a, b, c (and now all the predictions add up to 1% i.e. a, b, c = 1%), and the AC then takes a high and low bound 1-0 and takes the middle between the high and low, and starts misusing each % of the set, until matches the final letter in the window (same process whether compress or decompress). So say we stop once reach b in our set ex. a, *b*, c, we are in the float precision now of ex. 0.45-0.22. WE take middle again (0.23) and start misusing (once the window on the file takes another step. The encoding decimal keeps getting more precise, storing the whole file. To work in 16 byte float we need to carry away locked digits, meaning if the high and low are both now 0.457594-0.458988, we store '45' and get now 0.7594-0.8988, and we are going to be taking the middle of these 2 to make the decimal more precise then. This long decimal is then stored as a binary bin number ex. 6456453634636=10100011100111010011. I didn't implement the removing same counts from lower sets that are just from the higher set, because it hurt compression, i.e. if there is 9 counts total in set 3 and 99 total in set 2, 9 of the counts in set 2 are the same observations and 'should' not help us reach Roof. I'll look into it more. Lastly, escape letters, my first set we mix is a dummy set that has super small weight and has every possible letter, in case we need to encode/decode one and hasn't yet seen it in the file, hence requires a small room in the AC high low bounds. I also hardcoded each probability in this dummy set, common letters get more weight. Compression/decompression takes 2 hours and 16 minutes for 10MB, but Python is slower. Ram is fairly big because I didn't implement the pruning.
Title: Re: Possibly a breakthrough programming language?
Post by: LOCKSUIT on April 06, 2020, 03:31:03 AM
The algorithm searches before saves. And it only saves 1 line (17 letters long), not 17 saves.
Title: Re: Possibly a breakthrough programming language?
Post by: infurl on April 06, 2020, 03:33:08 AM
Have you considered using multiple passes on the data so your algorithm doesn't have to guess which sequences to compress first?
Title: Re: Possibly a breakthrough programming language?
Post by: LOCKSUIT on April 06, 2020, 04:12:59 AM
Usually it's not easy, the compressed data has much more noise (less patterns).
Title: Re: Possibly a breakthrough programming language?
Post by: LOCKSUIT on April 20, 2020, 10:49:58 PM
Since I wrote it up in full and clear and u may need it, here is actually how to make big money and hire AI/art/etc professionals from home. All AI domains are on there, it is pro-AI like heaven.
---------------------------------------------------------------------------------------------
Use Blockly to program in Python overnight, you don't need to learn Python. And Blockly has a language setting for blocks. On Upwork you can get easy 500USD for a single programming job.
- - - - -
Yes programming is really easy and I can do it now, and there is manyyyyy jobs on Upwork and all you need to do is make a good profile up cus like 1,000 apply to Upwork each day and you will be asked questions about Python in a one time interview. Find the easier but larger project that the client can't even understand (hence the large payment is common gesture) and it is just easy. Once you get the first job or two done, your profile will get boosted and more invites will be sent to you. You can invite too. Stay online as much as can and the clients will see you Online and choose you. I do that :) lol. I used to be a client but now I can code.

Upwork has all forms of AI domains, it's so funny lol. And, music, video design, blog posting, etc, any online job. So it's highly useful for making cash and hiring professional artists.

Just don't go off the Upwork cus they don't like that :). And you get a good review if nice and cooperative and on time if they need that, so make sure they give u good reviews! There is a way to legally leave Upwork with clients now btw!