Pretrained Transformer Pipelines and Task Selection

Learning outcomes

• Translate a product request into one defined NLP task.
• Run classification and extractive question-answering pipelines with explicit checkpoints.
• Select an evaluation and fallback that fit each output contract.

Set a task before choosing a checkpoint

“Use a transformer” is not a task definition. Start with an input, an output and a failure cost. A support team might need one label per message, entities with character spans, an answer copied from a supplied passage, a shorter version of a document or a translation into a named language. Those are different contracts and require different evaluations.

Read the model card before downloading a checkpoint. Check languages, labels, licence, base architecture, training data notes and known limitations. A high score on an unrelated benchmark does not establish fitness for your users.

Run two different task contracts

Create pipeline_contracts.py. The script uses explicit revisions where a production project should pin an immutable commit. Model downloads require an internet connection and several hundred megabytes of disk space.

python -m pip install "transformers>=4.45" torch
python pipeline_contracts.py

# pipeline_contracts.py
from transformers import pipeline

classifier = pipeline(
    "text-classification",
    model="distilbert/distilbert-base-uncased-finetuned-sst-2-english",
)
qa = pipeline(
    "question-answering",
    model="distilbert/distilbert-base-cased-distilled-squad",
)

review = "The update is useful, but installation took too long."
policy = "Refund requests are accepted within 14 days of purchase."

classification = classifier(review, truncation=True)[0]
answer = qa(question="How long is the refund window?", context=policy)

assert set(classification) == {"label", "score"}
assert {"answer", "score", "start", "end"}.issubset(answer)
assert policy:answer["end"]] == answer["answer"]

print("classification", classification)
print("answer", answer["answer"], "span", answer["start"], answer["end"])

What the output establishes

The classifier returns a task-specific label, while question answering returns a span tied to the supplied context. Scores are model confidence signals, not calibrated probabilities of correctness. Your numbers may vary with library and model revisions. The assertions check the result shape and the extractive span, not product quality.

Use a selection brief

Before a larger experiment, write five lines: user input, required output, unacceptable failure, evaluation set and fallback. Then compare a classical baseline, a frozen transformer and any adapted model on the same held-out examples.

• Use classification when labels are closed and stable.
• Use NER when positions and entity types matter.
• Use extractive QA when the answer must be grounded in a supplied passage.
• Use summarization only with factuality review appropriate to the risk.
• Use translation with fluent reviewers for the actual language pair and domain.

Consider latency, memory and batch size as part of model selection. A smaller checkpoint that meets the quality gate can be a better production choice than a larger model that exceeds the response budget.

Debug task mismatch

Knowledge check

Official sources and model references

• Transformers documentation
• Text classification task guide
• Model card documentation