NaturalNode
Named entity recognition
Do you have any plans for named entity recognition, I have seen that it would require a sequential classifier. The ability to train it with your own data set (json document) of POS tags and other key attributes.
Don't currently have a plan, if anyone wants to tackle this it would be great though!
Anyone have more thoughts on what algorithm might be best to implement for this?
@mbc1990 I think crf is the best model for NER, the pipeline is tokenize -> pos tag -> NER, the challenge is you need find a NER training data, it's a hard work.
Hi there everyone, I was just studying this subject and found some real interesting stuff about NER that I want to share:
- Vector Representations of Words
- Distributed Representations of Words and Phrases and their Compositionality
- Boosting Named Entity Recognition with Neural Character Embeddings
- a real good presentation
There some ways of doing this feature, the CharWNN seems to be the one with best results, but not by far. The others seems to need specific training corpus. For me it looks pretty similar to the PoS Tagger. I'm still not able to reproduce the algorithm detailed in those papers, also haven't found anything in javascript, only a few examples in python. Hope this will help to get any talented developer here inspired =)
I'm working on named entity recognition for natural. I'm working on three ways of recognition:
- regular expressions: can be used for time, date, uri's, currency, etc.
- vocabulary of named entities
- a (trained) model that classifies named entities
It will be possible to combine these approaches, so a hybrid approach. The methods returns a list of edges of the form (recognised string, start index, end index, category, score). Score only makes sense for the trained model. I'm thinking of using a maximum entropy model. Is that a viable route, any ideas on useful feature functions?
Hugo
Yes, I did some work on this: https://github.com/Hugo-ter-Doest/natural/tree/NER
I am trying to make a hybrid approach. First find the easy to define and match entities with regular expressions and lexicons, then apply a statistical model to do more advanced detection.
Hugo
Hi! Perhaps I can help with that, I did a NER but only for "enumerateds", with similar search, and my next step was to add regular expression entities (I see that you already had them!!! Great job!!!).
