Prasad Khake — Writing

The Vesuvius Challenge's July 2026 Progress Prizes named my Zarr 3 compatibility fix and chunk cache for the Python vesuvius library.

I tried to turn the best-scoring pockets from my PHerc1203 readability atlas into a readable surface.

A CPU-only survey of PHerc1203 found that most sampled sites fail a local geometry gate, while a few rare pockets initially looked comparable to known-readable PHerc1667.

I ran the public ink model on a 2.4 µm scan of sealed PHerc1203 and got speckle instead of letters.

I tested the public sarvam-translate model on three Windows 11 laptops with NVIDIA GPUs from 2014 and 2023.

There is no single best zarr chunk size for reading the Herculaneum scrolls.

The same idea — store attention keys and values in fewer bits — measured on two stacks: MLX's --kv-bits on an M3 raises RAM and slows decode 4×, while vLLM's fp8 KV cache on an L4 doubles capacity and speeds decode up to 28%.

A measured audit of Llama-3.1-8B on a rented L4: what vLLM's real, untouched default actually costs, what FP8 quantization gets you on top of it, and why a plausible-looking concurrency guess cost throughput instead of adding it.

Capping a model's KV cache instead of letting it grow without bound plateaus peak memory — but the instant a conversation outgrows the cap, recall of five planted facts collapses from 5/5 to 0/5, every trial, with zero partial credit.

John Carmack argued AI inference should stream model weights from cheap flash instead of expensive HBM, since weight access is deterministic.

I ran the same long-context test on a 16 GB fanless M3 and a ₹23 rented NVIDIA L4.

The applied chapter of a from-scratch project: after building tokenization, attention, gradient descent, a tiny GPT, and LoRA by hand, I ran a real QLoRA fine-tune — teaching Llama-3.2-1B to call tools on a MacBook, then measuring honestly what changed and

I wrote five GPU kernels from scratch on a 16 GB M3 to learn how LLM inference works at the metal.

Self-speculative decoding lets a model draft its own tokens by skipping layers — speculative decoding's speedup with no extra memory.

Atomic Chat advertises TurboQuant as 8x faster inference and 6x less memory.

Single-stream decoding on Apple Silicon is bottlenecked by reading the model's weights out of memory, not by the math.

LLaDA2.0-mini, a diffusion language model, runs on a 16 GB M3 and ties Qwen3-4B for the best answer-quality score I've measured (20/21).

A 1B draft model speeds up Llama-3.1-8B by 20% on an M3 — at num_draft_tokens=2.

On a 16 GB Mac, MLX's --kv-bits flag — whose entire job is to shrink the KV cache so longer contexts fit — raised peak memory at every context length I tested, and OOM'd at 32K where plain fp16 fit at 9.4 GB.

Apple shipped an official Python SDK for its on-device Foundation Model at WWDC 2026.

Google's Gemma-4 E4B posts the same math and factual scores as the full 12B on an M3 MacBook Air — in 6.6 GB instead of 11.4, at 8.2 tok/s instead of 2.7 — so on a 16 GB Mac the E4B is the one to run.

Re-running my own LLM benchmark, I found a bug that had inflated the quality scores in posts I'd already published.

On an M3 MacBook Air, Qwen3-4B with the thinking trace turned off scores 20/21 on a verifiable suite — beating Llama-3.1-8B's 18/21 at half the memory and nearly double the speed.

Google's Gemma 4 12B uses 11.4 GB of RAM and runs at 2.7 tok/s on an M3 MacBook Air — 2.4× the memory of Llama-3.1-8B at well under half the speed.

StreamingLLM (2023) found that keeping four specific tokens in the KV cache prevents catastrophic perplexity collapse at long contexts. mlx-lm implements this as RotatingKVCache(keep=4).

Benchmarking Gemma-3-12B, Qwen3-14B, and Llama-3.1-8B on a 16 GB MacBook Air (M3) with MLX.

Honest local-LLM benchmarks on a base M3, 16 GB — tokens/sec, peak RAM, and exactly where it hits the wall.

Debugging why tekken-v13 models emit Ġ instead of spaces through mlx-lm's server, and the one-line root cause in MLX's detokenizer routing.