NER Using Spacy model

Question

I continue to get the message that there are no NERs in my corpus. I am expecting that cats, dogs etc. will be identified as person. Let me know how to fix it.

import numpy as np
import pandas as pd

import spacy
from spacy import displacy

nlp = spacy.load("en_core_web_sm")

corpus=['cats are selfish', 'it is raining cats and dogs', 'dogs do not like birds','i do not like rabbits','i have eaten frogs snakes and alligators']

for sent in corpus:
    sentence_nlp = nlp(sent)
    # print named entities in sentences
    print([(word, word.ent_type_) for word in sentence_nlp if word.ent_type_])
    # visualize named entities
    displacy.render(sentence_nlp, style='ent', jupyter=True)

The error I get is:

[]
./NER_Spacy.py:19: UserWarning: [W006] No entities to visualize found in Doc object. If this is surprising to you, make sure the
Doc was processed using a model that supports named entity recognition, and check the `doc.ents` property manually if necessary
.
 displacy.render(sentence_nlp, style='ent', jupyter=False)
[]
./NER_Spacy.py:19: UserWarning: [W006] No entities to visualize found in Doc object. If this is surprising to you, make sure the
Doc was processed using a model that supports named entity recognition, and check the `doc.ents` property manually if necessary
.
 displacy.render(sentence_nlp, style='ent', jupyter=False)
[]
./NER_Spacy.py:19: UserWarning: [W006] No entities to visualize found in Doc object. If this is surprising to you, make sure the
Doc was processed using a model that supports named entity recognition, and check the `doc.ents` property manually if necessary
.
 displacy.render(sentence_nlp, style='ent', jupyter=False)
[]
./NER_Spacy.py:19: UserWarning: [W006] No entities to visualize found in Doc object. If this is surprising to you, make sure the
Doc was processed using a model that supports named entity recognition, and check the `doc.ents` property manually if necessary
.
 displacy.render(sentence_nlp, style='ent', jupyter=False)
[]
./NER_Spacy.py:19: UserWarning: [W006] No entities to visualize found in Doc object. If this is surprising to you, make sure the
Doc was processed using a model that supports named entity recognition, and check the `doc.ents` property manually if necessary
.
 displacy.render(sentence_nlp, style='ent', jupyter=False) ```

louis_guitton · Accepted Answer · 2019-10-06 05:53:24Z

I am expecting that cats, dogs etc. will be identified as person

You're not expecting the right thing then :) Spacy's models for NER are trained on different datasets depending on the language. In the case of the model you're using see here : https://spacy.io/models/en#en_core_web_sm

The dataset used to train the model you're using is called "Onto Notes 5" and that one doesn't consider cats and dogs as PERSON (as most people do). If you want to get "cats" and "dogs" as entities, you need to train your own NER model with your own data. For example you could label some data with the ANIMAL entity using regex rules with a list of pets of interest, and using that labelled dataset, you can fine tune the NER model to do what you want.

Collectives™ on Stack Overflow

NER Using Spacy model

1 Answer 1

Comments

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Related