Building & Production
Shipping AI features: cost, latency, reliability, and operations.
9 Application
- Your First LLM API Call From zero to a working request, plus the four errors that catch everyone on the way.
- How Do You Know It Works? Without evals you are tuning prompts by vibes. A dozen saved examples is enough to start, and it changes how you work.
- From Demo to Production A working demo is maybe a fifth of the way there. Five gaps separate something that works when you try it from something that works when everyone does.
- Which Model Should You Use? Match the model tier to the task rather than defaulting to the largest. A framework that survives the next model release.
- Cutting Your API Bill Four places cost hides, ordered by how much they usually save. Most bills are dominated by one fixable thing.
- Why Responses Stream Token-by-token delivery is not a UI trick. It is the raw shape of generation, and it changes perceived latency more than any optimization.
17 Production
- Latency Optimization Perceived speed is dominated by time to first token, and most of that is prefill. Where the milliseconds actually go.
- Observability for LLM Systems When a user reports a bad answer next week, you need to reconstruct what happened. What to log, and the one field everyone omits.
- Cost Monitoring and Attribution Token spend is easy to measure and easy to ignore until the bill arrives. What to track, and which dimension actually explains your costs.
- Prompt Versioning Prompts are code that changes behaviour. Treat them accordingly, and keep the eval result attached to the version.
- A/B Testing AI Features Nondeterministic output and subjective quality break the usual experiment design. What to measure instead, and why offline evals come first.
- Migrating Between Models New model, better benchmarks, worse output. Prompts are tuned to models, and migration is a real project rather than a config change.
- Reliability Patterns Providers rate-limit, time out, and return malformed output. The patterns that keep a system up when its dependency is unreliable.
- Self-Hosting Models Running open-weight models on your own hardware. When it makes sense, what it actually costs, and the arithmetic to do first.