Search text for 999999999999999999 words in no time

  • 3 Replies
  • 186 Views
*

LOCKSUIT

  • Emerged from nothing
  • Trusty Member
  • *****************
  • Sentinel
  • *
  • 3550
  • First it wiggles, then it is rewarded.
Search text for 999999999999999999 words in no time
« on: September 09, 2019, 02:24:48 AM »
Here's one of the things I just made. Try it out. I tested it on Windows. You can search for any word in a huge amount of data really fast as if it is a 1 word search. Runs on CPU. You can swap the 200MB in the src folder. Run it in Visual Studio 2019. You can edit the words I search for in main.cpp line 199.

This is useful for when you have many many items you want to search for and you have a large amount of data to look through.

https://www.dropbox.com/s/v9vxy1bhpogppkq/FastSearch.rar?dl=0
Emergent

*

LOCKSUIT

  • Emerged from nothing
  • Trusty Member
  • *****************
  • Sentinel
  • *
  • 3550
  • First it wiggles, then it is rewarded.
Re: Search text for 999999999999999999 words in no time
« Reply #1 on: September 09, 2019, 04:02:41 AM »
RAM actually shrinks to ~150MB to hold ~220MB.

Binary Search would make RAM go up lots. The data stays as Huffman words.

So, it's fast+low memory.
« Last Edit: September 09, 2019, 08:18:11 AM by LOCKSUIT »
Emergent

*

LOCKSUIT

  • Emerged from nothing
  • Trusty Member
  • *****************
  • Sentinel
  • *
  • 3550
  • First it wiggles, then it is rewarded.
Re: Search text for 999999999999999999 words in no time
« Reply #2 on: September 09, 2019, 08:14:35 AM »
There's a bug wait, it will be even faster. It's already fast.
Emergent

*

LOCKSUIT

  • Emerged from nothing
  • Trusty Member
  • *****************
  • Sentinel
  • *
  • 3550
  • First it wiggles, then it is rewarded.
Re: Search text for 999999999999999999 words in no time
« Reply #3 on: September 10, 2019, 11:11:35 AM »
btw if not clear: I put the search items in a tree instead of putting the data in a tree. The little tree searches the data.

update:
I made it faster:
https://www.dropbox.com/s/5lym9vl2fttrmgd/FasterFastSearch.rar?dl=0
268MB becomes 132MB and takes 12 seconds to check the full 132MB for any of 72 words (could have set it to ex. 99999 words). It takes 12 seconds no matter how many words are being searched for.
Emergent

 


Now you see it...
by Art (General Chat)
Today at 02:12:23 AM
Can your PC run it?
by LOCKSUIT (Gaming)
October 22, 2019, 06:47:43 PM
advancements on the spider pully leg design
by goaty (Home Made Robots)
October 22, 2019, 04:41:24 PM
Who are you, what do you do & what do you want to do ?
by LOCKSUIT (New Users Please Post Here)
October 22, 2019, 03:49:37 PM
XKCD Comic : Wardrobe
by Tyler (XKCD Comic)
October 22, 2019, 12:00:23 PM
Consciousness & Self-awareness
by PhilNewAGI (General AI Discussion)
October 21, 2019, 01:44:00 PM
beyond omega level coding
by yotamarker (General AI Discussion)
October 21, 2019, 10:19:49 AM
Invert music....
by goaty (General Chat)
October 21, 2019, 07:34:48 AM

Users Online

11 Guests, 1 User
Users active in past 15 minutes:
AndyGoode
[Trusty Member]

Most Online Today: 28. Most Online Ever: 340 (March 26, 2019, 09:47:57 PM)

Articles