Production & Ops
Run AI at scale: serving, observability, cost control, and safe iteration.
For MLOps and platform engineers · 10 articles · ~13.3h · 3 stages
This track assumes you already know
- 1
Serve and scale
- How Inference Serving WorksBatching, scheduling, and memory management. Why throughput and latency pull against each other, and what a serving system is actually doing.Advanced Architecture
- Self-Hosting ModelsRunning open-weight models on your own hardware. When it makes sense, what it actually costs, and the arithmetic to do first.Production
- Latency OptimizationPerceived speed is dominated by time to first token, and most of that is prefill. Where the milliseconds actually go.Production
- 2
Observe and control
- Observability for LLM SystemsWhen a user reports a bad answer next week, you need to reconstruct what happened. What to log, and the one field everyone omits.Production
- Cost Monitoring and AttributionToken spend is easy to measure and easy to ignore until the bill arrives. What to track, and which dimension actually explains your costs.Production
- Reliability PatternsProviders rate-limit, time out, and return malformed output. The patterns that keep a system up when its dependency is unreliable.Production
- 3
Iterate safely
- Prompt VersioningPrompts are code that changes behaviour. Treat them accordingly, and keep the eval result attached to the version.Production
- A/B Testing AI FeaturesNondeterministic output and subjective quality break the usual experiment design. What to measure instead, and why offline evals come first.Production
- Migrating Between ModelsNew model, better benchmarks, worse output. Prompts are tuned to models, and migration is a real project rather than a config change.Production
- Statistical Significance: Why One Comparison DeceivesModel A scored 84%, model B scored 86%. Is B better, or did it get lucky on your test set? Significance testing separates a real difference from noise.Evaluation