Capstone: Build a Production-Ready Data Mining System

Capstone outcomes

Choose one low-risk problem: delayed-order investigation, equipment fault review, catalogue recommendation or support-ticket routing. Use synthetic or lawfully reusable data.

• Create a versioned project brief, data contract and responsible-use review.
• Build idempotent preparation and an analytical table with reconciliation.
• Compare a baseline with one justified mining method under deployment-matched evaluation.
• Package prediction or discovery output, monitoring, rollback and portfolio evidence.

One coherent system beats a collection of notebooks

Start with the action and stop conditions from Module 1. Define the unit, population, cutoff, outcome or discovery target, capacity and review owner. State what the system will not do.

Create raw, staged, accepted and rejected layers. Record checksums and run IDs. A rerun must not duplicate facts or recommendations. Keep schema validation and reconciliation executable.

Choose the method from the task. Classification needs labelled outcomes and prospective evaluation. Clustering needs geometry and stability evidence. Association and sequence mining need support and later confirmation. Recommendation needs temporal ranking and coverage checks.

Add a simple interface such as a command-line report or local web page that shows result, evidence, review status and limitations. Do not expose personal data, secrets or unsupported explanations.

Monitoring covers input schema, missingness, drift, score or pattern distribution, system errors, human review outcomes and performance when labels mature. Define a rollback that returns to a baseline rule or disables the output safely.

Create a machine-readable release gate

This small validator turns capstone requirements into a release decision. It does not replace human approval, security review or domain validation.

Block release when evidence is missing

checks = {
    "brief_reviewed": True,
    "source_rights_recorded": True,
    "raw_data_preserved": True,
    "pipeline_idempotent": True,
    "reconciliation_passed": True,
    "baseline_beaten_on_final_test": False,
    "subgroup_review_completed": True,
    "security_review_completed": True,
    "monitoring_and_rollback_tested": True,
}

failed = [name for name, passed in checks.items() if not passed]
decision = "BLOCKED" if failed else "READY FOR OWNER REVIEW"

print("release decision:", decision)
print("failed checks:", failed)
if failed:
    print("next action: investigate evidence; do not weaken the gate to force release")

Expected blocked decision

release decision: BLOCKED
failed checks: ['baseline_beaten_on_final_test']
next action: investigate evidence; do not weaken the gate to force release

A blocked model can still be a successful learning project. The honest result is more valuable than a deployed system whose evidence failed.

Mandatory failure exercises

Demonstrate recovery rather than writing ‘handled’ in a checklist.

• Corrupt an input schema and show a clear, non-destructive failure.
• Rerun a completed load and prove target counts do not change.
• Simulate drift or a new category and show monitoring output.
• Trigger rollback and verify users receive the declared safe fallback.

Complete and present the capstone

Your final walkthrough should let a reviewer challenge each claim and reproduce each result without private credentials.

• Publish architecture, data lineage and threat or privacy review.
• Include baseline, final evaluation, error or pattern analysis and rejected alternatives.
• Provide one-command local setup with synthetic sample data.
• Record limitations, maintenance owner, refresh triggers and next evidence needed.

Final portfolio package

• Runnable repository and tests.
• Decision-focused technical report.
• Short demo that includes a failure and rollback, not only a success path.

Official references and further reading

• NIST AI Risk Management Framework (Official risk-management framework)
• scikit-learn common pitfalls (Official leakage and consistency checks)
• PostgreSQL transaction tutorial (Official safe data-change foundation)
• Apache Spark monitoring (Official scalable job monitoring guidance)

Review note for Capstone: Build a Production-Ready Data Mining System: recheck the linked documentation after a dependency changes the relevant API, metric or modelling assumption, then record the tested version beside your result.