[Ultron]

PROJECT J.A.R.V.I.S.
Just a rather very intelligent system

This project aims at creating a program advanced enough to compete with it's fictional counterpart from the Iron Man franchise. Please note that this program is not expected to grow into an Artificial Intelligence (according to my definition of the term) however most people will likely consider it as such.

Currently the program has entered the second phase where the focus is on improving the original code and concepts. So far it can only reply to a small set of precisely written sentences due to it's small 'vocabulary' (database) and also behaves like a grammar Nazi.



The program is designed to make use of three different databases:

• AwDB - contains a list of words with their corresponding 'types'. This one is used to identify the 'meaning' of the words found in the input string;
• BwDB - contains the same items (words) as AwDB, however structured like a nest in a reverse order. This one is used to find the appropriate words when preparing the output (feedback);
• pDB - an ever-growing table containing all the sentence patterns the program has encountered so far. It is updated with new ones automatically. It is used to find matching reply patterns which are later filled with the help of BwDB.

The most basic pseudo-description of the loop sequence would look something like this:

• Read input (inString);
• Extract words to string array;
• Find each word in AwDB and extract type field;
• Return string (inPat, pattern) with ordered types (noun, verb, noun, adj);
• Find pattern string (inPat) in pDB and return output pattern(outPat);
• Use BwDB to replace the 'types' in outPat with appropriate words and for output string (outString);

Note that this is the original idea and design of the program's main loop. It has been subject to many changes and has since complicated and grown radically.

The hard part is that it is not a question of whether or not the program will work, yet will it be truly intelligent in the way it was meant to be? This means that each bug finding will not be as easy as reading compiler error messages and may in fact take some time.

What is the goal?
The current goal is to prove some concepts of functionality and create a program much like Marvel Studio's character (program) which can control hardware by recognizing auditory commands and replying with such. It is also supposed to provide an accurate chatbot platform however with the goal of answering with accurate data (computer status, house etc.). So the aim is not to seem human yet to be a grammatically correct chatter and assistant with the ability to learn how to reply to (at a given time in the past) unknown sentences and forms.

Download latest setup (0.1A): https://www.dropbox.com/s/79cc4jopuithnor/setup.exe?dl=0

Download source (0.2): https://www.dropbox.com/s/yd5zizmato3l0xo/Version%200_2.zip?dl=0

Stay tuned!

P.S. Please share your opinion about Jarvis here: http://aidreams.co.uk/forum/index.php?topic=7717.0#.VOXQD9LF8sE

[Ultron]

UPDATE 11.02.2015




Only one function and 32 errors to go! Considering this is the first attempt to compile the program I believe I am doing pretty good and within the week I should be able to post the first screenshots of the working program!


Sadly, It seems that I will have to either manually fill the databases with words or create secondary programs to do that for me - both variants are to be equally challenging and boring. Please note that there are to be approximately 2 - 2.5 million entries in total. Currently I am exploring the idea of organizing the data in distinct tables, not databases. Whatever I decide to do will surely be subject to change in the future as the current database system is not flexible enough.


Anyway, the only remaining function is the one that replaces the pattern array elements with actual words, thus forming the automated reply. This one is bound to be the most challenging and I am looking forward to spending the weekend solving the problems I encounter with it.

[Ultron]

UPDATE 13.02.2015

It did not take long to isolate the causes and fix the errors. Honestly, I was expecting somewhat of a challenge with those but oh well - SQLite rewarded me with another one. Specifically, I have not been able to find a way to store the values returned from the databases into a variable (string). The functions provided by sqlite3.h execute the SQL commands you specify and return 0 - the information is sent to stdout (standard output - console).


 I have also been unable to find a workaround and might have the edit the source code for the database system itself.





On the positive side I have finally finished the 'FOF' function, which is the key to deciding the system's reply. I am already aware of the possible logical errors it is going to make and have already made concepts for improving and fixing those errors accordingly.


It is interesting how these things turn out - the program should remain fairly similar to the initial version (with several technical tweaks), while the logical improvements are to affect the databases - how and which data they store. In the next update the program will be able to better identify different words - currently 'chair' and 'Hitler' are all the same to the output processor.


Expecting to officially finish 0.1A by the end of the weekend. After that I am preparing to put this project on a short hold as I am also working on a new nanomaterial for a competition beginning in March.

[Ultron]

UPDATE 14.02.2015

I have found away through the database problem - it turns out that sqlite3_exec, while convenient and cleaner, is also a very limited function. I am now using it's core sub-functions to retrieve raw data from the database (sqlite3_prepare_v2)  and can now easily store it in variables. There is no more need for callback functions, too.
Here is how the code looks: http://prntscr.com/64y657


Also I am now using DB Browser for SQLite and no longer have to code a whole program in order to add a few extra elements to my tables.
You can find it for free at: http://sqlitebrowser.org/

[Ultron]

UPDATE 15.02.2015

I have wasted yet another day on fixing trivial technical errors that cause other 'logical' errors. The program can currently access the databases, execute queries and find out the pattern of the input sentence provided by the user. However issues such as fundamental type conversion prevent the last function to finish compiling the output. While there is a lot of data and answers on StackOverflow regarding C++ type conversion I still need to combine a lot of methods just to retrieve one variable from the database and it is different every time because the program structure is very specific.



You can take a sneak-peak of some of the code as well as how the database is basically organized here: http://prntscr.com/656tb7


Also, since the database structure and content will change frequently as the program's concept advances and becomes more 'accurate', I have decided to name every update to the database system in order to keep track more easily. The current version is officially jDS I.

[Ultron]

UPDATE 16.02.2015

This program has not yet even made it through it's main function and it is already hellbent on destruction! I mean, just look at all those type conversions - not even the compiler can keep track of them anymore!





It is what I have been working on the whole weekend - type conversions. The last function - specifically, the sub-segment that compiles the output - was full of tiny problems visible only through a compiler. Many different functions (mostly those coming from SQLite) require a variety of different types for mostly the same values, and as a result the program now is at least twice the size it was originally intended to be, possibly to have repercussions on speed (which would not be obvious until the databases all completely filled).


On the bright side, there is some progress - I have managed to repair and tidy up everything down to the last 15-20 lines. Originally the program compiled with 35 errors and 12 warnings - now there are only 3 warnings. Also, since the last update I have managed to organize and print out the string containing the output pattern (NCTA on image), but for some unusual reason I am unable to extract the characters from it (N, C, T, A) which are required to find corresponding words to place in the final reply (output string).
This is weird to me as I have done it many times before in the program and I have tried all of the methods suggested by the C++ community.


This might take a while...

[Ultron]

UPDATE 19.02.2015

All the type conversions have been handled, all the functions have been written and the code has been tested to be A OK! Except for this one little bugger, which took some 4-5 days to find...



So for some reason I can only fetch the value of this string ONCE...

Until recently I believed the problem was with my string separation functions (I had used a previously tested method, new methods including string streams, vectors etc..) and the all didn't work because there wasn't any value to the parameter variable passed to them. How cool is that.

I believe SQLite3 had some warning about accessing variables after the database connection was closed, however I had not encountered this problem with the previous 3 database access points.

On the bright side, the rest of the code seems to be working properly and It does not crash when it hits the last 'for' loop. Can't wait to fix this up and get started on improving the program! Also I am exploring the idea of creating a grammar processing engine (library) for C/C++. However I see no way to incorporate it into Jarvis since it is also meant to 'understand', in a way, what the user is saying (what he wants). It will require me to brush up on linguistic morphology though.


Sub-Update #1


It would seem that I have accidentally fixed this bug while trying to pinpoint the error line. However, the output is now more mystifying than ever...





I understand there is an undesired looping somewhere, but the weirdest part is that I do not have a single 'cout' or variable containing a value of "amazing" (bottom right).


Sub-Update #2

I found out that the mysterious 'amazing' came from the return word database because it was of type 'A' and 'A' was the only type (char) that the program extracted from the output pattern string.




As you can see I have solved that problem and the problem correctly extracts the 'type' characters from the string (and in the correct order). The next step (and obstacle) is to convert the character vector containing them into a string vector. The compiler spots a problem in some core MinGW libraries, however the real reason is that the function that is supposed to do that can only take 'char*' and not 'char'.

There might be a more efficient way to break a string (such as "N C T A") and store the elements individually into a string vector, but keep in mind I don't (and don't intend to) get paid for this lol. One step at a time, though - progress has been constant so far so I am not worried about failure.


Sub-Update #3

I did some tests on a separate .cpp file so I could focus and keep better track. Turns out I can cut off roughly 25-35 lines of code from the FOF() function. Look at that simplicity:





I stumbled upon a nice, not-so-highly voted answer on Stack Overflow. Line 22 is the key to many gates. Will attempt to implement this into my program tomorrow - got all Friday night to spend on it.

[Ultron]

UPDATE 20.02.2015

I realize I am starting to look pathetic - spending a whole week in attempts to correct a few technical mistakes regarding type conversion, but what can you do. At least I finally witnessed my program forming (or at least starting to form) his own reply. Specifically, he output the first word of the reply he was predicted to (his database is so small).



I was referring to the word 'friend'...

This is because, as you can see, only the first character (word type) of the output string <pattern> has been copied correctly. The others have been 'poisoned' with other characters and the program cannot locate a word type of <shuriken> " in it's output word database.
I am uncertain about to cause however I am more than certain that it will give me a headache.


The whole code that causes and precedes the cause is captured in the image, so you can thank me for the free puzzle.

[Freddy]

Guys, please discuss Ultron's project in the discussion thread here :

http://aidreams.co.uk/forum/index.php?topic=7717.0

[Ultron]

UPDATE 21.02.2015 - VERSION 0.1A

Jarvis now officially lives! I have managed to fix all of the trivial errors and for the first time ever I have witnessed his 'reply' in all of it's artificial glory!
The last edit I made was changing the types of 1 array and 1 variable that the last stringstream in function 'FOF' used. Specifically, I changed them both from type 'char' to type 'string' - something I originally did not think would work.



Of course it requires some tweaking...

However the real journey is yet to begin... There is a countless number of functions and updates I need to do before this can actually be used or even 'tested' by others. But I still need to fix one more little problem in there and finish commenting the code before I can release the source.
Before I continue on improving it and move on to Version 0.1B and later 0.2 I need to take a few day brake in order to think through the problems and obstacles it is bound to encounter and how to fix them. I also have other projects and civil duties I must fulfill this week before I continue on.

Download project: https://www.dropbox.com/s/vmbajt67h0n31qa/Version%200_1A.zip?dl=0

Note: You might notice a non-standard header called "Console.h" - you can learn about it here: http://www.codeproject.com/Articles/9130/Add-Color-to-Console-Projects

[Ultron]

UPDATE 01.03.2015 - VERSION 0.2

There was not much coding work to be done in this update, but quite a hell of thinking. The transition to jDB II has been successful and I have not encountered any logical errors so far.







The only affected database was AwDB, specifically there are now more word types and a 'preceded' column. As for the code, there is a new function that finds the position of the word. It's function is commented in the source code itself.


There is also a new feature - some mini programs (tools) that one mind find useful in their own chatterbot projects. I use them to automate filling and emptying the databases as a much faster alternative to DB Manager, which requires manual input. The programs are easy to edit and you may feel free to improve on their code (they can be much more efficient) and re-distribute your improved version of it.
Currently I do not suggest using them to fill a database with over 100k entries. Also, if you are inputting strings containing a ' or " you might have a problem.


This update was aimed at easing the transition towards future versions instead of having a direct and radical shift towards the next structure. It is still not usable however now it is a more robust platform which you can take as an example for certain projects.


The next version is going to be able to execute commands and will contain even more types of words - these will be specialized types which you don't find in your average English book. Also there will be a new tool which accesses uses sockets. Version 0.4 will aim towards implementing TTS and finally have a complete word database of at least 100 thousand words.


Download project: https://www.dropbox.com/s/yd5zizmato3l0xo/Version%200_2.zip?dl=0

[Ultron]

VANEHEIM ENCRYPTION ALGORITHM

I have been working on an encryption algorithm and developing a new cryptography concept for quite some time now, however it wasn't until recently that I decided to actually code a program by it. I suppose the decision was provoked by a better understanding of the risks involved with my projects. Also I do not trust conventional encryption protocols related to computers and server security since I have seen these easily broken by the government.





I will of course NOT share the source code, as it would render the whole program useless. However I do plan on sharing the .exe and an installation. The program includes an option to encrypt, decypt or even 'scramble' a file (WIP feature) which would render the file 'eternally crypted'.


I guess this is my way of approving my inactivity related to the Jarvis project, although Vaneheim will officially be a part of Version 0.3 as a Tool.


And regarding the main project... I have decided to skip a few 'concept versions' of the jDB system architecture and move on directly to a brand new jDB VI which is currently in the works. After version 4 I plan on releasing an actually usable program - version 1.0.

[Art]

Very cool project. There have been many attempts at cryptography, PGP, AES, DES, ECC, EFS, RC4 and many other types of cyphers and methods to encrypt and decrypt files and secrets. Also steganography, hidding messages, files, etc., within another images or document. Quite interesting: http://en.wikipedia.org/wiki/Steganography

A very enlightening site can be found here: http://www.garykessler.net/library/crypto.html
and from our friends at NSA, the National Cryptologic Museum, which I often pass on my way to and from work!:
http://www.nsa.gov/about/cryptologic_heritage/museum/

Good luck on your project!! How long would you estimate it would take for a competent hacker to break your code? How long for the good folks at the NSA to do the same? 

[Ultron]

Well this differs from all other encryption algorithms by 1 little thing, which can be a great advantage but also a great disadvantage. The advantage is in that that 'thing' makes it unique and exponentially harder to break, while the disadvantage is that it sort of disables real-time encrypted communication (or makes it harder, at least). Also, for one to decrypt it he must be aware of the 'parameter' used just to pass the first 'stage'.

Anyway I have encountered stenography previously in my early 'hacker' carrier at hackthiswebsite - there is a specific set of challenged which requires more then the average computer hacker. I have only passed the first challenge and have not attempted the others in years...

P.S. I suppose you know the drill with these 2 replies

[Art]

Yeah, how about we put on a different record? LP perhaps?