Named Entity Recognition with Span-Level Evaluation

Evaluate Entity Spans Instead of Counting Easy Matches

• Extract entity text, character offsets and labels from a document.
• Measure exact-span matches and review partial overlaps separately.
• Define labels with examples and counterexamples before collecting annotations.
• Review exact and partial span errors separately before changing labels or model rules.

Spans are the product interface

Offsets connect a prediction to evidence and make corrections possible. Normalised values belong in separate fields.

Overlapping entities require an explicit policy because many sequence labelling models return non-overlapping spans.

Precision and recall tell different stories

High recall suits candidate generation followed by human review. High precision may matter for automatic redaction, where false positives remove legitimate text.

Annotation quality sets the ceiling

Two annotators can disagree about subsidiaries, titles or nested names. A written guideline makes those disagreements visible.

Split documents, not sentences, when adjacent text from one source would leak into both train and test sets.

Review confusion by label

Inspect missed entities, wrong labels and boundary-only errors separately. Each category suggests a different fix.

A larger model will not repair an inconsistent label definition.

Build the Named Entity Recognition with Span-Level Evaluation example

This single sentence is a sanity check, not an evaluation. Replace expected with reviewed spans across representative documents.

Character offsets are measured against the original Python string. Any cleanup before inference can shift them.

Install:

python -m pip install spacy
python -m spacy download en_core_web_sm

import spacy
nlp = spacy.load('en_core_web_sm')
text = 'Ada Lovelace worked with Charles Babbage in London.'
doc = nlp(text)
for ent in doc.ents:
    print(ent.text, ent.start_char, ent.end_char, ent.label_)
expected = {('Ada Lovelace', 'PERSON'), ('Charles Babbage', 'PERSON'), ('London', 'GPE')}
predicted = {(e.text, e.label_) for e in doc.ents}
print('exact correct:', len(expected & predicted), 'of', len(expected))

Expected Spans and Evaluation Counts

Ada Lovelace 0 12 PERSON
Charles Babbage 25 40 PERSON
London 44 50 GPE
exact correct: 3 of 3

Exact matching is intentionally strict. Maintain a separate overlap report to diagnose near misses without inflating the production metric.

Never use a general entity model as an automatic personal-data detector without testing the relevant data types and languages.

Diagnose failures in Named Entity Recognition with Span-Level Evaluation

Compare predicted and gold spans by offsets, then inspect the surrounding sentence.

Check your Named Entity Recognition with Span-Level Evaluation reasoning

Check your span-evaluation judgement before opening the explanations. Look for boundary errors, label disagreement and partial matches.

Primary references for Named Entity Recognition with Span-Level Evaluation

• spaCy linguistic features: official tags, dependencies and entity guidance.
• spaCy rule-based matching: use matchers to create explicit span baselines for reviewed entity cases.
• Hugging Face dataset cards: record entity labels, annotation gaps and licensing for the span dataset.

Revalidate this lesson when the spaCy pipeline or annotation guidelines change. Record the model package and reviewed span set used for comparison.