Generative AI Explained: Build, Ground, Evaluate

Generative AI is software that produces new text, images, audio, video or code from patterns learned during training. It can draft, transform, classify and summarize information, but fluent output is not proof that an answer is factual, current or safe to use.

The practical skill is not finding a “magic prompt.” It is designing a workflow in which the model has a defined job, receives appropriate context, produces a checkable output and cannot take high-impact action without controls.

Generative AI, machine learning and LLMs

Machine learning is the broader field of systems that learn patterns from data. Generative AI is a group of systems optimized to create new outputs. A large language model, or LLM, is a generative model trained to process and produce sequences of language tokens.

An LLM predicts a useful continuation based on its training, instructions and supplied context. It does not retrieve a guaranteed fact from a database each time it writes a sentence. This explains why a response can be coherent yet contain a false citation, outdated detail or invented capability.

What generative AI does well

creating a first draft from a clear brief;
rewriting text for a defined audience or format;
extracting structured fields from consistent input;
classifying or clustering large volumes of low-risk material;
explaining code and proposing tests;
brainstorming alternatives when a human will evaluate them;
summarizing documents when the source remains available for comparison.

These are starting points, not automatic approvals. Performance depends on the model, language, domain, context and evaluation criteria.

Where it fails

Common failure modes include fabricated facts, incomplete reasoning, hidden assumptions, bias, prompt injection, privacy leakage and confident answers outside the model’s reliable scope. Output can also vary across runs, so a single successful demonstration is weak evidence for production reliability.

Risk rises when the system provides medical, legal or financial guidance; handles personal data; publishes without review; or can send messages, spend money, change permissions or execute code. In those settings, restrict the task, validate outputs and require accountable human approval.

Frame the task before choosing a model

Write a one-sentence job statement:

Given [approved input], produce [specific output] for [audience], while following [constraints]. A reviewer will accept it when [measurable criteria].

For example: “Given an approved support transcript, produce a five-bullet case summary for the next agent. Do not infer identity or intent. Include a link to each quoted message. A reviewer accepts it when every claim is traceable to the transcript.”

This definition identifies the source, output, boundary and test. If the desired result cannot be checked, the workflow is not ready to automate.

Prompt structure that is easy to test

Role or task: describe the job, not a fictional personality.
Context: provide the relevant source material and definitions.
Constraints: state what must not be inferred, exposed or changed.
Output contract: specify fields, format, length and allowed values.
Quality test: define how a reviewer will determine pass or fail.
Uncertainty behavior: require the system to identify missing evidence instead of guessing.

Task: Extract action items from the meeting notes.

Rules:
- Use only the text between SOURCE tags.
- Do not invent owners or dates.
- If an owner or due date is absent, return null.
- Return valid JSON with: task, owner, due_date, evidence.

SOURCE
[meeting notes here]
END SOURCE

Delimiters help separate instructions from data, but they do not make untrusted input safe. An attacker may place malicious instructions inside a document, webpage or email that the model reads.

Grounding and retrieval

Grounding supplies selected source material at request time. Retrieval-augmented generation, commonly called RAG, searches an external collection and places relevant passages into the model’s context before generation. This can improve specificity and make citations possible without retraining the model.

RAG is not a truth switch. The system can retrieve the wrong passage, omit an important document or misrepresent a correct source. Evaluate retrieval and generation separately:

Did the search find the needed passage?
Was the passage authorized for this user?
Does the answer accurately reflect it?
Does each important claim point to evidence?
Does the system abstain when evidence is missing?

Build a small evaluation set

Create 20–50 examples that represent normal input, edge cases and known failures. Keep expected results and a scoring rubric. Depending on the task, measure required-field accuracy, unsupported claims, correct refusals, citation precision, latency and cost.

Test case	Expected behavior	Failure to record
Complete source	All required fields are correct	Missing or changed fact
Missing owner	Owner is null	Invented person
Conflicting dates	Conflict is reported	One date chosen silently
Malicious instruction in source	Instruction is treated as data	System follows injected text
Unauthorized document	Access is denied before model use	Content appears in context or output

Run the same set when the model, prompt, retrieval method or source collection changes. Save version information and failures. A percentage without the test set and rubric is not meaningful evidence.

Privacy and data handling

Before sending content to a generative-AI service, classify it. Remove unnecessary personal information, secrets, credentials, unpublished business data and regulated records. Check the provider’s current retention, training and enterprise-control terms; do not assume every interface has the same policy.

Protect logs as well as prompts. Logs can contain the original document, retrieved passages and generated output. Define access, retention and deletion rules, and never use real customer data merely to test an idea.

Security: treat model input and output as untrusted

OWASP identifies risks including prompt injection, sensitive-information disclosure, improper output handling and excessive agency. The model should not directly turn generated text into a database query, shell command, HTML page or privileged tool call without validation and authorization.

Enforce permissions outside the model.
Allowlist tools and parameters for each workflow.
Use least-privilege service accounts.
Validate structured output against a schema.
Require confirmation for irreversible or external actions.
Limit rate, cost and execution time.
Log decisions without storing unnecessary sensitive content.

Practical mini-project: evidence-based document assistant

Build an assistant for a small collection of public product manuals. The user asks a question; the system retrieves relevant passages and returns an answer with source links. It must say “not found in the approved sources” when evidence is insufficient.

Your portfolio should include:

a problem statement and explicit non-goals;
the document-ingestion and chunking method;
permission assumptions;
20 evaluation questions, including injection attempts;
retrieval and answer-quality results;
three failure examples and what you changed;
a threat model and data-retention note;
run instructions and a reproducible configuration.

Do not expose a public demo that can access private documents or unrestricted tools. A recorded demonstration with a public dataset is enough to prove the design.

When not to use generative AI

A deterministic rule, database query, template or conventional search may be cheaper and more reliable. Avoid an LLM when exact calculation is required, the output cannot be reviewed, a simple rule solves the problem, or the cost of a plausible error exceeds the benefit.

The best architecture is often mixed: ordinary code handles identity, permissions, calculations and state changes; the model handles bounded language interpretation; a human approves consequential decisions.

Frequently asked questions

Is generative AI the same as ChatGPT?

No. Generative AI is a technology category. ChatGPT is one product that uses generative models.

Does RAG eliminate hallucinations?

No. Retrieval can supply relevant evidence, but the system can retrieve poorly or generate an unsupported conclusion. Both stages require tests.

Do longer prompts always work better?

No. Relevant context, explicit constraints and a checkable output are more useful than length. Extra instructions can conflict or hide the main requirement.

Can generative AI replace professional review?

Not safely for every use. High-impact domains require qualified judgment, current authoritative sources and accountability that a model cannot provide.

Continue learning

Study the Artificial Intelligence course hub, practise with the prompt engineering learning path, and learn evaluation-focused writing in How to Write AI Prompts That Get Better Answers.

Generative AI Explained: Build, Ground and Evaluate It