MetaCyberGuru Academy
Trace attention from its core calculation to pretrained pipelines and careful fine-tuning, with evaluation that separates a successful training run from a useful model.
Module result: Project: Compare Five Transformer NLP Tasks.
Why this module belongs in the course
Transformer families differ in training objective and architecture. Encoder-only, decoder-only and encoder-decoder are starting points for reasoning, not rigid rules for every model.
Before you begin
The concepts and project evidence from Module 13. You should also be able to create a Python virtual environment and keep private or employer data out of the exercise.
Four lessons, one connected result
- Lesson 1Self-Attention, Subword Tokenization and Transformer Families85 min · Advanced
- Lesson 2Pretrained Transformer Pipelines and Task Selection75 min · Intermediate
- Lesson 3Fine-Tuning, PEFT, LoRA and Quantization95 min · Advanced
- Lesson 4Project: Compare Five Transformer NLP Tasks120 min · Advanced
How to know you are ready to continue
Complete the checkpoint without copying the worked example. Keep the code, output and a short decision note. Your note should explain one choice, one failure you observed and one limitation a reviewer should know.
Primary references for this module
The lessons explain the ideas in original wording. Use these primary or official sources when a library interface, standard or research claim needs verification.
- https://arxiv.org/abs/1706.03762
- https://huggingface.co/docs/tokenizers/
- https://huggingface.co/docs/transformers/
- https://huggingface.co/docs/hub/model-cards
- https://huggingface.co/docs/transformers/tasks/sequence_classification
- https://huggingface.co/docs/peft/
- https://huggingface.co/docs/bitsandbytes/
- https://huggingface.co/docs/evaluate/
Share this page
Share this page with the people who will use it next.
Discussion
No comments yet. Add the first useful question or observation.
You must log in to post a comment.