NLP and Data Mining Final Assessment and Capstone

Part 1: defend the foundations

Answer these questions in your own words. Use one concrete example for each answer.

1. How can fitting a preprocessor before a train and test split leak information?
2. When can precision matter more than recall, and when can the reverse be true?
3. Why does a high-confidence association rule not prove that one item causes another?
4. What makes a cluster useful beyond a high silhouette score?
5. How is novelty detection different from finding outliers inside a training dataset?
6. What information can aggressive text cleaning destroy?
7. Why should a text classifier be compared with a simple TF-IDF baseline?
8. What does a retrieval metric measure that an answer-quality metric does not?
9. How can a retrieved document attack an RAG system?
10. Which changes would make you roll a deployed model back?

Part 2: diagnose three broken systems

A classifier that looks too good

A team reports 99 percent accuracy. The dataset contains repeated customer records, the target is rare, and the split was random. Describe the tests you would run before believing the result. Your answer should consider duplicates, grouped splitting, class imbalance, baseline performance and a metric tied to the cost of mistakes.

A topic model nobody can use

An NMF model returns ten topics with top words, but reviewers cannot give the topics stable names. Explain how you would inspect preprocessing, choose the topic count, test stability, compare document clusters and decide whether the project should use search or supervised labels instead.

A confident RAG answer with the wrong source

The answer sounds correct but cites an outdated policy. Trace the possible failures through ingestion, metadata, chunking, retrieval, reranking, context assembly and generation. Then state which tests and monitoring signals should prevent the same failure from reaching users.

Part 3: final capstone

Choose one problem that needs both pattern discovery and language processing. Two suitable examples are a customer-feedback intelligence system and a document discovery platform. You may choose another topic if its data is lawful to use and the work has comparable depth.

Required repository structure

capstone/
├── README.md
├── DATASET.md
├── MODEL_CARD.md
├── requirements.txt
├── data/
├── notebooks/
├── src/
├── tests/
├── reports/
└── app/

Required evidence

1. Problem brief: user, decision, input, expected output, failure cost and a measurable definition of useful.
2. Dataset record: source, licence, collection date, sampling, fields, label process, exclusions and known limitations.
3. Data audit: missing values, duplicates, imbalance, suspicious leakage and at least five inspected examples.
4. Baseline: a rule, frequency baseline or simple model that sets a minimum standard.
5. Advanced approach: one justified mining or NLP method, not an algorithm chosen only because it is fashionable.
6. Evaluation: task metrics, error categories, subgroup or slice checks, latency and resource notes.
7. Failure report: at least five high-impact failures from different error categories, each with a diagnosis and a decision to fix, accept or reject the approach. Add more examples when a category keeps recurring.
8. Safety review: privacy, copyright, bias, access control, injection or poisoning risks that apply to the project.
9. Working interface: a command-line tool, notebook report or small web API that demonstrates the complete flow.
10. Operations note: versioning, monitoring, drift checks and rollback criteria.

Capstone review rubric

When to call the course complete

Use the rubric honestly. A course-complete project has no unknown data source, no hidden test-set tuning, no unexamined critical failure and no deployment claim without a monitoring plan. If a row remains in “Needs revision”, repair that part and record what changed. That revision is part of the learning, not evidence of failure.

Back to the complete course roadmap