Natural Language Processing library

  • 6 Replies
  • 388 Views
*

infurl

  • Trusty Member
  • ********
  • Replicant
  • *
  • 605
  • Humans will disappoint you.
    • Home Page
Natural Language Processing library
« on: April 28, 2018, 10:31:55 pm »
Found this relatively new natural language processing library this morning. It's only a statistical parser so it might not provide much new functionality, but it is written in C with Python bindings and it claims to be fast. The examples given show a nice clean API. Has anyone tried it out yet?

https://spacy.io/

*

spydaz

  • Trusty Member
  • ******
  • Autobot
  • *
  • 230
  • Developing Conversational AI (Natural Language/ML)
    • SpydazWeb
Re: Natural Language Processing library
« Reply #1 on: April 29, 2018, 12:34:21 am »
Em .....


I've seen it!.....
Most of their functions i have already created .... But their neural network looks interesting... Although I'm not ready to use neural networks in my AI yet.

I was able to reduce my database by Creating a data-warehouse then extracting the data-marts required. obviously there were lots of Crossover information as well as duplicate data. Things like the opencyc data has no actual use as it is full of bad information.
but when I created my parts of speech tagging the main dictionary was created. Over time i adjusted my Tagging system adding Various categories and lists such as "cars" so when a car is detected the tagger tags it as PNNC (PronounCARS) or Colors as PNNCL. this enables for Categorical entity recognition. as these needs increase or lists are created I often delete tags and Re-Enter under a New-Tag... Requirements / needs of NLP are constantly changing.  I do not use the REPLACEMENT TECHNIQUE .... so my database actually only contains "facts".

If i was to tell the AI Arts owns a CAR... It would store the information in a new "person" location alongside its Predicate (Owns),  Person (owns) OBJECT.... When Enquiring about Art owning a Car it would check His person object for the properties known about that person.... if known...(BASIC).

the route that i have chosen is not to collect all information or attempt to program every answer.... I rely on NLP and Grammar to do All the Work.... (i only have the Topic/Response Keyword/Detect Topic... QA ... Paradigm remaining from the past (obviously and old QADB TOO) as it works well for Loose learning as well as targeted data capture.

The NLP librarys out there are mainly for statistical modelling as its the current trend..... The R programming language has created the "BUZZ" although everybody prefers to Use the python for these tasks. Well done to Tensorflow and such interfaces (commercial)
The Spacy is actually a simple product to use.... Although (Limited by their Small Database) !

*

infurl

  • Trusty Member
  • ********
  • Replicant
  • *
  • 605
  • Humans will disappoint you.
    • Home Page
Re: Natural Language Processing library
« Reply #2 on: April 29, 2018, 12:42:12 am »
Um thanks I think.

*

spydaz

  • Trusty Member
  • ******
  • Autobot
  • *
  • 230
  • Developing Conversational AI (Natural Language/ML)
    • SpydazWeb
Re: Natural Language Processing library
« Reply #3 on: April 29, 2018, 01:29:17 am »
Found this relatively new natural language processing library this morning. It's only a statistical parser so it might not provide much new functionality, but it is written in C with Python bindings and it claims to be fast. The examples given show a nice clean API. Has anyone tried it out yet?

https://spacy.io/

Really im saying that The API is Good but these librarys are not coming up with anything NEW just repackaging.... I suppose the best API will win!

*

infurl

  • Trusty Member
  • ********
  • Replicant
  • *
  • 605
  • Humans will disappoint you.
    • Home Page
Re: Natural Language Processing library
« Reply #4 on: April 29, 2018, 01:42:04 am »
Really im saying that The API is Good but these librarys are not coming up with anything NEW just repackaging.... I suppose the best API will win!

Thanks, that's actually what I thought you were saying and I agree with that too. I was just distracted by the free-form nature of your reply. ;)

The only advantage this one claims to have is speed and interchangeable models. There is a third party advertising (link spamming) models for it on chatbots.org today which is how I found out about it. Though less relevant, it also claims to be based on a neural network architecture.

In terms of what it can do, it is inferior to my own parsing libraries in all areas so I'm not considering using it myself. I thought others might be though.

*

Don Patrick

  • Trusty Member
  • *******
  • Starship Trooper
  • *
  • 453
    • Artificial Detective
Re: Natural Language Processing library
« Reply #5 on: April 29, 2018, 08:04:52 am »
I haven't tried it but I've come across a lot of NLP people recommending it as much as Stanford's parser, so I reckon it's as solid as statistical parsers will get.
Personal project: NLP -> learning -> knowledge -> logical inference -> A.I.

*

spydaz

  • Trusty Member
  • ******
  • Autobot
  • *
  • 230
  • Developing Conversational AI (Natural Language/ML)
    • SpydazWeb
Re: Natural Language Processing library
« Reply #6 on: May 06, 2018, 11:49:07 am »
I haven't tried it but I've come across a lot of NLP people recommending it as much as Stanford's parser, so I reckon it's as solid as statistical parsers will get.

Its a Starting point! its quite old now.

But really to build your own... you just need your dictionary / and Context free grammar to find the unknowns..... (or implement a hidden Markov sequencing prediction).... then you may find it works better! (Controlled is always better) as any errors you can deal with them... There is also a Porter Stemmer which is out there.... Also a Good starting point.

 


Users Online

41 Guests, 3 Users
Users active in past 15 minutes:
infurl, ivan.moony, squarebear
[Trusty Member]

Most Online Today: 52. Most Online Ever: 208 (August 27, 2008, 09:36:30 am)

Articles