LLM Applications, RAG, Evaluation and Security

Why this module belongs in the course

A convincing LLM answer is not evidence of a reliable application. Retrieval, citations, access control, prompt-injection resistance and repeatable evaluation belong in the system design.

Before you begin

The concepts and project evidence from Module 14. You should also be able to create a Python virtual environment and keep private or employer data out of the exercise.

Four lessons, one connected result

1. Lesson 1Structured LLM Outputs, Decoding and Tool Boundaries80 min · Advanced
2. Lesson 2RAG Ingestion, Hybrid Retrieval, Reranking and Citations95 min · Advanced
3. Lesson 3Evaluate Retrieval and LLM Answers with Calibrated Rubrics90 min · Advanced
4. Lesson 4Project: Build a Secure, Cited RAG Assistant125 min · Advanced

How to know you are ready to continue

Complete the checkpoint without copying the worked example. Keep the code, output and a short decision note. Your note should explain one choice, one failure you observed and one limitation a reviewer should know.

Primary references for this module

The lessons explain the ideas in original wording. Use these primary or official sources when a library interface, standard or research claim needs verification.

• https://json-schema.org/draft/2020-12/json-schema-core
• https://genai.owasp.org/llm-top-10/
• https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf
• https://arxiv.org/abs/2005.11401
• https://www.postgresql.org/docs/current/textsearch-controls.html
• https://github.com/pgvector/pgvector
• https://huggingface.co/docs/evaluate/
• https://docs.ragas.io/en/stable/concepts/metrics/available_metrics/