title: Evaluation in a Spacy NER model
body: <p>I am trying to evaluate a trained NER Model created using <a href="https://spacy.io/docs/usage/training-ner" rel="noreferrer">spacy lib</a>.
   Normally for these kind of problems you can use f1 score (a ratio between precision and recall). I could not find in the documentation an accuracy function for a trained NER model. </p>
  
  <p>I am not sure if it's correct but I am trying to do it with the following way(example) and using <code>f1_score</code> from <code>sklearn</code>:</p>
  
  <pre><code>from sklearn.metrics import f1_score
  import spacy
  from spacy.gold import GoldParse
  
  
  nlp = spacy.load("en") #load NER model
  test_text = "my name is John" # text to test accuracy
  doc_to_test = nlp(test_text) # transform the text to spacy doc format
  
  # we create a golden doc where we know the tagged entity for the text to be tested
  doc_gold_text= nlp.make_doc(test_text)
  entity_offsets_of_gold_text = [(11, 15,"PERSON")]
  gold = GoldParse(doc_gold_text, entities=entity_offsets_of_gold_text)
  
  # bring the data in a format acceptable for sklearn f1 function
  y_true = ["PERSON" if "PERSON" in x else 'O' for x in gold.ner]
  y_predicted = [x.ent_type_ if x.ent_type_ !='' else 'O' for x in doc_to_test]
  f1_score(y_true, y_predicted, average='macro')`[1]
  &gt; 1.0
  </code></pre>
  
  <p>Any thoughts are or insights are useful. </p>