Ai Dreams Forum

Member's Experiments & Projects => AI Programming => Topic started by: LOCKSUIT on September 09, 2019, 02:24:48 am

Title: Search text for 999999999999999999 words in no time
Post by: LOCKSUIT on September 09, 2019, 02:24:48 am

Here's one of the things I just made. Try it out. I tested it on Windows. You can search for any word in a huge amount of data really fast as if it is a 1 word search. Runs on CPU. You can swap the 200MB in the src folder. Run it in Visual Studio 2019. You can edit the words I search for in main.cpp line 199.

This is useful for when you have many many items you want to search for and you have a large amount of data to look through.

https://www.dropbox.com/s/v9vxy1bhpogppkq/FastSearch.rar?dl=0

Title: Re: Search text for 999999999999999999 words in no time
Post by: LOCKSUIT on September 09, 2019, 04:02:41 am

RAM actually shrinks to ~150MB to hold ~220MB.

Binary Search would make RAM go up lots. The data stays as Huffman words.

So, it's fast+low memory.

Title: Re: Search text for 999999999999999999 words in no time
Post by: LOCKSUIT on September 09, 2019, 08:14:35 am

There's a bug wait, it will be even faster. It's already fast.

Title: Re: Search text for 999999999999999999 words in no time
Post by: LOCKSUIT on September 10, 2019, 11:11:35 am

btw if not clear: I put the search items in a tree instead of putting the data in a tree. The little tree searches the data.

update:
I made it faster:
https://www.dropbox.com/s/5lym9vl2fttrmgd/FasterFastSearch.rar?dl=0
268MB becomes 132MB and takes 12 seconds to check the full 132MB for any of 72 words (could have set it to ex. 99999 words). It takes 12 seconds no matter how many words are being searched for.