Ziraph blog

Apple fm is the newest runner I added to ziraph, and the odd one out: it hides its token count, runs on the Neural Engine instead of the GPU, and quietly trades speed for power.

Where your inference runs is a margin, privacy, and battery decision, and every upside hinges on whether your model actually works on the user's hardware - the aspects an engineering leader should weigh before betting on on-device AI.

No new chips at WWDC 2026 - but a structural rebuild of Apple's on-device AI: Core AI replaces Core ML, the matmul moved into the GPU, and the flagship cloud model runs on NVIDIA GPUs in Google Cloud.

Every local AI tool prints tokens per second; none of them prints what those tokens cost.

A matched-quant MLX vs raw llama.cpp benchmark for Gemma 4 12B on one M1 16GB: decode is a tie - both pinned at the memory-bandwidth wall.

I wanted GPU die temperature, and every SMC key came back 0x84, key not found - even the ones that could not be wrong.

anesight was a system monitor for the Apple Neural Engine - the wrong part of the chip to watch.