ai-llm

AI / LLM Integration

How to build features on top of large language models — feeding them your own data, letting them call your code, and checking that the answers are actually good.

9 units·41 lessons·~39 h

Start track →

Start from zero

Before the senior material: what LLM integration even is, and the handful of words the rest of the track assumes you already know.

01 Start from zero: what LLM integration actually is 10 min

Prompt caching

Roadmap

Prompt caching: how a stable prefix cuts input cost 10x

Prompt caching reuses an exact token-for-token prefix at ~0.1x the input price. Any byte that shifts that prefix — a…

Funding: 49% ETA: when funded Back this unit →

01 Prompt caching: how a stable prefix cuts input cost 10x 16 min 02 Prompt caching: multiple-choice review 13 min 03 Prompt caching: free-recall review 14 min 04 Prompt caching: request and usage reading 14 min 05 Prompt caching: measure the savings 210 min

Tool calls

Roadmap

Tool calls: the round-trip loop, schema validation, and the guard against runaway agents

Tool calling turns a model into a function caller, but the model only emits a request — you execute it. Every call is a…

Funding: 64% ETA: when funded Back this unit →

01 Tool calls: the round-trip loop, schema validation, and the guard against runaway agents 17 min 02 Tool calls: multiple-choice review 13 min 03 Tool calls: free-recall review 14 min 04 Tool calls: code and schema reading 14 min 05 Tool calls: build a robust tool-calling loop 240 min

Rag architecture

Roadmap

RAG architecture: the pipeline that fails at retrieval, not generation

RAG is a retrieval problem wearing a generation costume. Chunk size, top-k, reranking, and context order each move…

Funding: 52% ETA: when funded Back this unit →

01 RAG architecture: the pipeline that fails at retrieval, not generation 17 min 02 RAG architecture: multiple-choice review 13 min 03 RAG architecture: free-recall review 14 min 04 RAG architecture: code and pipeline reading 14 min 05 RAG architecture: build and evaluate a retrieval pipeline 240 min

Streaming

Roadmap

Streaming LLM responses: SSE, partial tokens, and the proxy that eats them

Streaming trades total generation time for time-to-first-token. SSE emits delta events you accumulate; tool-call args…

Funding: 57% ETA: when funded Back this unit →

01 Streaming LLM responses: SSE, partial tokens, and the proxy that eats them 16 min 02 Streaming: multiple-choice review 13 min 03 Streaming: free-recall review 14 min 04 Streaming: code and stream reading 14 min 05 Streaming: build a robust streaming endpoint and client 240 min

Cost budgets

Roadmap

LLM cost budgets: token asymmetry, routing, and the kill switch

Output tokens cost ~5x input, and the system prompt + history + RAG context get re-sent every turn — so a runaway loop…

Funding: 49% ETA: when funded Back this unit →

01 LLM cost budgets: token asymmetry, routing, and the kill switch 17 min 02 LLM cost budgets: multiple-choice review 13 min 03 LLM cost budgets: free-recall review 14 min 04 LLM cost budgets: code and cost arithmetic 14 min 05 LLM cost budgets: build cost observability and guardrails 240 min

Agents

Roadmap

The agent loop: ReAct, runaway steps, and context that grows every turn

An LLM agent is a while-loop that calls the model, runs a tool, appends the result, and calls the model again. The…

Funding: 43% ETA: when funded Back this unit →

01 The agent loop: ReAct, runaway steps, and context that grows every turn 17 min 02 Agents: multiple-choice review 13 min 03 Agents: free-recall review 14 min 04 Agents: code and loop reading 14 min 05 Agents: build a bounded, observable agent 240 min

Evals

Roadmap

LLM evals: the regression test for non-deterministic features

You can''''t ship an LLM feature without evals: the same input gives different output, and a model or prompt change…

Funding: 44% ETA: when funded Back this unit →

01 LLM evals: the regression test for non-deterministic features 17 min 02 LLM evals: multiple-choice review 13 min 03 LLM evals: free-recall review 14 min 04 LLM evals: code and harness reading 14 min 05 LLM evals: build an eval suite and CI gate 240 min

Putting it together

Roadmap

Composing a production LLM app: the bug lives in the seam

Caching, RAG, streaming, tools, agents, and evals each pass their own tests, then fail together. Trace one request end…

Funding: 48% ETA: when funded Back this unit →

01 Composing a production LLM app: the bug lives in the seam 17 min 02 Composing LLM apps: multiple-choice review 13 min 03 Composing LLM apps: free-recall review 14 min 04 Composing LLM apps: code and trace reading 14 min 05 Capstone: ship a composed LLM feature that holds at the seams 240 min

Build with this track

Guided projects that exercise what you learn here.

◆ Projects

Grounded RAG Service

A RAG demo that answers from a corpus is easy; a RAG service you'd trust in front of users is not. The hard part isn't retrieval, it's grounding: making the model say only what the retrieved text supports, attaching citations the reader can check, and proving with an eval set that the answers don't drift into confident fiction. You'll build the whole loop — chunk, embed, store, retrieve top-k, ground, cite, score — and feel exactly where it leaks.

Next track

Engineering Practice

The habits that separate a hobby project from a professional team — testing well, shipping changes safely, and keeping a service healthy in production.