AI Engineering in 76 Minutes (Complete Course/Speedrun!)

Marina Wyss - AI & Machine Learningautoenpublicupdated Jul 28, 2026, 11:34

Read in about 5 minutes instead of watching 76 minutes.

AI Engineering and Foundation Models

AI engineering focuses on building applications with existing foundation models, emphasizing adaptation rather than training models from scratch.
Foundation models learn through self-supervision and increasingly support multimodal inputs such as text, images, and video.
Transformers use attention to reference relevant tokens and process inputs in parallel, although long contexts increase computation and memory costs.
Scaling improves capabilities but faces diminishing returns, limited high-quality training data, and growing electricity requirements.

Supervised fine-tuning teaches conversational behavior, while preference fine-tuning aligns responses with human preferences through methods such as RLHF and DPO.
Temperature, top-k, and top-p sampling control the balance between deterministic accuracy and creative diversity.
Probabilistic generation explains output inconsistency and confident hallucinations.

Evaluation is difficult because AI tasks are often open-ended, subjective, expertise-intensive, and poorly represented by saturated public benchmarks.
Useful evaluation methods include functional correctness, exact match, lexical or semantic similarity, human review, and AI judges.
AI judges are scalable but can be inconsistent and exhibit self, position, and verbosity biases.
Select models by filtering hard constraints such as licensing and privacy, then comparing domain capability, instruction following, cost, and latency on application-specific tests.
Evaluation metrics should connect directly to business outcomes, cover important user and data segments, and be tested for reliability.

Prompt engineering adapts model behavior without changing weights and should be treated as a systematic experimental process.
Effective prompts combine a clear task description, representative examples, the concrete request, and an explicit output format.
Complex tasks can be decomposed into subtasks, paired with reasoning or self-critique, and optimized through versioned experiments.
Production systems must defend against prompt extraction, jailbreaking, prompt injection, and sensitive-information extraction.
Security measures include adversarial testing, isolated execution, human approval for impactful actions, anomaly detection, and input and output guardrails.

RAG supplies query-specific external information through a retriever and a foundation-model generator.
Retrieval may use fast lexical search, semantic embedding search, or hybrid pipelines that retrieve candidates and rerank them.
Chunk size, overlap, query rewriting, metadata augmentation, and reranking must be tuned for the application and data.
RAG also supports images and structured data, including systems that translate natural-language questions into SQL.

Agents observe an environment, plan decisions, and use tools such as search, databases, calculators, code interpreters, APIs, and write actions.
Multi-step agents face compounding errors and higher risk because their tools can affect external systems.
Planning should be separated from execution, validated before tool calls, and reviewed by humans for sensitive tasks.
Agent evaluation should measure plan validity, tool and parameter correctness, completion rate, step count, latency, and cost.
Agent memory combines model knowledge, the short-term context window, and external long-term stores such as RAG.

Fine-tuning is most useful after prompting has been exhausted, especially for domain specialization, structured outputs, or stronger task-specific behavior.
Use RAG for information failures and fine-tuning for behavioral failures; combining both can produce the strongest results.
Quantization, gradient checkpointing, and parameter-efficient methods reduce the substantial memory required for fine-tuning.
LoRA trains small low-rank matrices while freezing original weights, enabling efficient adaptation with relatively little data.
High-quality, relevant, consistent, diverse, compliant, and correctly formatted data often matters more than sheer volume.
Start with roughly 50 to 100 carefully designed examples, measure improvement, and expand the dataset only when results justify it.

Inference optimization balances cost, latency, throughput, utilization, and the application's responsiveness requirements.
Identify whether workloads are compute-bound or memory-bandwidth-bound before choosing hardware and optimizations.
Key latency measures include time to first token and time per output token; percentile measurements are more informative than averages.
Common model optimizations include quantization, pruning, distillation, speculative decoding, and more efficient attention.
Service-level techniques include dynamic or continuous batching, prompt caching, decoupled prefill and decode, and multiple forms of parallelism.

AI applications commonly evolve from a direct model call toward context construction, guardrails, model routing, gateways, caching, and controlled write actions.
A model gateway provides a unified interface, access control, fallbacks, load balancing, cost management, logging, and analytics.
Monitoring detects failures, while observability records enough internal state to diagnose their causes.
Explicit and implicit user feedback creates proprietary data that can drive continuous product and model improvement.
Architecture complexity should solve real problems; simpler systems are often more reliable and maintainable.

Define application-specific evaluation criteria, usefulness thresholds, and business-linked metrics before adapting a model.
Filter model candidates by hard constraints, use public benchmarks only for shortlisting, and run your own representative experiments.
Design a standard internal model interface so providers and models can be swapped without rewriting the application.
Version prompts, track experiments with standardized datasets and metrics, and store prompts separately from application code.
Validate agent plans before execution and require human approval for sensitive or impactful actions.
Start adaptation with prompting, add RAG for missing information, and use fine-tuning when behavioral problems remain.
Pilot fine-tuning with 50 to 100 high-quality examples before investing in a large dataset.
Preserve original data, inspect distributions and disagreements, deduplicate examples, remove noncompliant content, and use the correct model template.
Profile inference bottlenecks and optimize against latency, throughput, cost, and utilization requirements.
Collect detailed logs and connect component-level operational metrics to business outcomes.