Search text for 999999999999999999 words in no time

  • 3 Replies
  • 138 Views
*

LOCKSUIT

  • Emerged from nothing
  • Trusty Member
  • ****************
  • Admiral
  • *
  • 3457
  • First it wiggles, then it is rewarded.
Search text for 999999999999999999 words in no time
« on: September 09, 2019, 02:24:48 am »
Here's one of the things I just made. Try it out. I tested it on Windows. You can search for any word in a huge amount of data really fast as if it is a 1 word search. Runs on CPU. You can swap the 200MB in the src folder. Run it in Visual Studio 2019. You can edit the words I search for in main.cpp line 199.

This is useful for when you have many many items you want to search for and you have a large amount of data to look through.

https://www.dropbox.com/s/v9vxy1bhpogppkq/FastSearch.rar?dl=0
Emergent

*

LOCKSUIT

  • Emerged from nothing
  • Trusty Member
  • ****************
  • Admiral
  • *
  • 3457
  • First it wiggles, then it is rewarded.
Re: Search text for 999999999999999999 words in no time
« Reply #1 on: September 09, 2019, 04:02:41 am »
RAM actually shrinks to ~150MB to hold ~220MB.

Binary Search would make RAM go up lots. The data stays as Huffman words.

So, it's fast+low memory.
« Last Edit: September 09, 2019, 08:18:11 am by LOCKSUIT »
Emergent

*

LOCKSUIT

  • Emerged from nothing
  • Trusty Member
  • ****************
  • Admiral
  • *
  • 3457
  • First it wiggles, then it is rewarded.
Re: Search text for 999999999999999999 words in no time
« Reply #2 on: September 09, 2019, 08:14:35 am »
There's a bug wait, it will be even faster. It's already fast.
Emergent

*

LOCKSUIT

  • Emerged from nothing
  • Trusty Member
  • ****************
  • Admiral
  • *
  • 3457
  • First it wiggles, then it is rewarded.
Re: Search text for 999999999999999999 words in no time
« Reply #3 on: September 10, 2019, 11:11:35 am »
btw if not clear: I put the search items in a tree instead of putting the data in a tree. The little tree searches the data.

update:
I made it faster:
https://www.dropbox.com/s/5lym9vl2fttrmgd/FasterFastSearch.rar?dl=0
268MB becomes 132MB and takes 12 seconds to check the full 132MB for any of 72 words (could have set it to ex. 99999 words). It takes 12 seconds no matter how many words are being searched for.
Emergent

 


Open AI - Hide & Seek - OMG COOL!
by Art (General AI Discussion)
Today at 12:52:25 pm
XKCD Comic : Prescience
by Tyler (XKCD Comic)
September 17, 2019, 12:02:32 pm
TRANSFORMERS
by LOCKSUIT (General AI Discussion)
September 17, 2019, 07:42:09 am
THIS is Bayesian
by WriterOfMinds (General AI Discussion)
September 17, 2019, 04:25:22 am
Share your most unique powerful AGI digests
by AndyGoode (General AI Discussion)
September 16, 2019, 01:29:46 am
Efficient vectors ....
by LOCKSUIT (General AI Discussion)
September 15, 2019, 06:09:03 am
A Pleasant Mind
by LOCKSUIT (General AI Discussion)
September 15, 2019, 03:08:10 am
Fractal Orbit
by ivan.moony (Human Computer Interaction)
September 14, 2019, 04:13:07 pm

Users Online

17 Guests, 1 User
Users active in past 15 minutes:
ivan.moony
[Trusty Member]

Most Online Today: 43. Most Online Ever: 340 (March 26, 2019, 09:47:57 pm)

Articles