Karthika Raghavan

Part 6 of this series was a landscape post — no new experiments, just a close read of why prefill stops being compute-bound once prefix caching does its job well enough.

The premise that broke We normally treat prefill as compute-bound and decode as memory-bandwidth-bound.

Let me be honest about how this post happened, because it’s a better story than the post itself.

The LLM Serving Stack You Actually Need: From NGINX to llm-d to GPU A layer-by-layer guide for DevOps, SRE, and infrastructure engineers building scalable, secure, production-grade LLM services — and wondering why the naive approach keeps falling over.

On April 21, 2026, Grafana Labs quietly dropped a blog post: AI Observability for Agents in Grafana Cloud.

This post is the honest account of what happened when I ran experiments 5 and 6 — the P/D disaggregation guide from the llm-d repo — on a second Lambda Labs GH200 instance.

In the previous post I documented everything that broke during the llm-d deployment on a Lambda Labs GH200.

Let me set expectations before we start.

I spent a week running inference experiments on my M4 Mac Mini — not to build a product, but to understand what actually happens inside an LLM serving stack.

Let me be honest with you.

Translating SQL to NoSQL: Architecture Deep-Dive Part 1 of 2: Design decisions, trade-offs, and algorithms behind schema-travels Why I Built This Migrating from a relational SQL database to a NoSQL paradigm is notoriously difficult to automate.

Building a Privacy-Aware LLM Gateway: Benchmarking Results Part 2 of 2: Empirical evaluation of classification accuracy, routing performance, and cost attribution Abstract In Part 1, I described the architecture of inference-sentinel, a privacy-aware LLM…