Project: Build a Data Discovery Brief That Can Be Reviewed

Definition of done for this checkpoint

A strong brief does not predict that the project will succeed. It makes success, failure and uncertainty inspectable. Use a question you understand, such as delayed orders, support-ticket routing or equipment faults.

• Write a decision-focused objective and name the person or team who could act.
• Define the unit, population, feature cutoff, outcome and baseline.
• Inventory proposed sources with provenance, permissions and join risks.
• Set evaluation, subgroup, privacy and deployment rejection gates.

A useful brief has tensions, not just fields

The objective should describe an action, not ‘use AI’. ‘Rank tomorrow’s eligible repair jobs so two technicians can inspect the ten highest-risk machines’ gives you a population, a time horizon and a capacity. It also exposes the cost of a false alarm and a missed failure.

The data section needs more than column names. State who owns each source, how often it changes, which join key connects it, and whether the join could duplicate rows. Record when a feature becomes available. A maintenance note entered after failure cannot be used for an earlier prediction.

Your baseline should be hard to misunderstand. It might be the current manual rule, the majority class, a seasonal average or a simple logistic regression. The proposed mining approach earns its complexity only if it improves on that baseline in a way the decision maker values.

Risk gates are concrete stopping conditions. Examples include an unusable label, missing permission, too few positive cases, unstable performance on a later time period or unacceptable errors for a relevant subgroup. Writing them before results appear makes the review more credible.

Finish with an evidence plan. List the files, charts and decisions that will be saved. Reproducibility is easier when it is designed into the work rather than reconstructed at the end.

Create and validate the brief as JSON

JSON forces every important decision to have a named place. Save the code as validate_brief.py. It writes discovery_brief.json only after the required fields and timeline pass simple validation.

Turn the project contract into a file

import json
from datetime import date

brief = {
    "project": "Next-day equipment fault review",
    "decision_owner": "maintenance supervisor",
    "unit": "machine-day",
    "eligible_population": "active machines with a full prior day of sensor data",
    "feature_cutoff": "2026-06-30",
    "outcome_start": "2026-07-01",
    "outcome_end": "2026-07-07",
    "baseline": "rank by count of threshold alarms in the prior day",
    "primary_metric": "precision among the top 10 ranked machines",
    "sources": [
        {"name": "sensor_hourly", "owner": "operations", "join_key": "machine_id"},
        {"name": "repair_log", "owner": "maintenance", "join_key": "machine_id"},
    ],
    "stop_conditions": [
        "repair timestamps cannot be reconstructed reliably",
        "source use is not approved",
        "later-period precision does not beat the current rule",
    ],
}

required = {
    "project", "decision_owner", "unit", "eligible_population", "feature_cutoff",
    "outcome_start", "outcome_end", "baseline", "primary_metric", "sources",
    "stop_conditions",
}
missing = required - brief.keys()
if missing:
    raise ValueError(f"Missing fields: {sorted(missing)}")

cutoff = date.fromisoformat(brief["feature_cutoff"])
start = date.fromisoformat(brief["outcome_start"])
end = date.fromisoformat(brief["outcome_end"])
if not cutoff < start <= end:
    raise ValueError("The feature cutoff must precede the outcome window")
if len(brief["stop_conditions"]) < 2:
    raise ValueError("Add at least two independent stop conditions")

with open("discovery_brief.json", "w", encoding="utf-8") as handle:
    json.dump(brief, handle, indent=2)

print("Brief valid")
print(f"Decision owner: {brief['decision_owner']}")
print(f"Sources recorded: {len(brief['sources'])}")
print(f"Stop conditions: {len(brief['stop_conditions'])}")

Checkpoint output

Brief valid
Decision owner: maintenance supervisor
Sources recorded: 2
Stop conditions: 3

A successful run checks structure, not truth. The maintenance supervisor and source owners still need to review whether the definitions match reality. Add their questions to the brief instead of quietly editing history.

How to review the brief without defending it

Ask a reviewer to try to break the plan. The goal is to discover uncertainty while it is cheap. These failures are useful outcomes.

• If the decision owner cannot describe a different action, the prediction may become an unused dashboard.
• If labels arrive weeks later, note that monitoring and retraining cannot use immediate ground truth.
• If joins change the row count, inspect one-to-many relationships before accepting the dataset.
• If a stop condition cannot be measured, rewrite it as an observable threshold or required approval.

Complete the module checkpoint

Replace the example with your own low-risk project. Use synthetic or clearly licensed public data if the source has personal, confidential or operationally sensitive records.

• Run the validator and commit both the code and generated JSON.
• Draw the feature and outcome timeline in a README.
• Ask one person to review the definitions and record at least three objections.
• Revise the brief and keep the first version so the decision trail remains visible.

Checkpoint evidence

• A valid discovery_brief.json with no secrets or personal data.
• A README explaining the decision, baseline, metric and timeline.
• A review log with objections, responses and unresolved specialist questions.

Official references and further reading

• From Data Mining to Knowledge Discovery in Databases (Foundation for the discovery lifecycle used in the brief)
• NIST AI Risk Management Framework (Official framework for mapping and managing AI risks)
• Python json documentation (Primary reference for the checkpoint file format)

Review note for Project: Build a Data Discovery Brief That Can Be Reviewed: recheck the linked documentation after a dependency changes the relevant API, metric or modelling assumption, then record the tested version beside your result.