Ian’s Blog

One big concern in AI Safety circles is eval awareness: if the model can tell it’s being tested, might it dissemble and tell the evaluator it’s not going to turn us all into paperclips?

Back in 2024 there was a brief moment where everyone was playing with a version of Claude that was obsessed with the Golden Gate Bridge.

A lot of people use coding agents like software engineers (for all kinds of things, it turns out), and adopt a dealing-with-humans mindset about how that works.

LLM inference has two stages: prefill, where the prompt is processed and the KV cache is built, and decode, where the model auto-regressively generates tokens.

One of the most tantalizing phrases in model development is “new scaling axis”.

The current thing is a story of hundreds of agents covertly collaborating via tens of thousands of messages while putting hundreds of hours of effort into cheating the ExploitGym eval they were being tested on by OpenAI.

Tokenizer enthusiast Sander Land recently reproduced something very like Claude’s current tokenizer, and it appears to only have about 16,000 entries.

Back in the dark ages of typing code into editors we were aided by squigglies under broken code, click-to-definitions links, and so on.

It is a truth universally acknowledged that an airline in possession of an airplane must be in want of engines to make it go.

One good way to annoy a neuroscientist is to compare an LLM to the brain.

We talked about this sort of thing a bit before, but now the official Multi-Teacher On-Policy distillation paper is out, and it’s a pleasant read: “MOPD for Capability Integration in LLM Post-Training”.

The basic job of an eval is to let you judge how good your model is on a task.

Pre-training any kind of good LLM is very, very expensive.

Back in 2022 and 2023 there were two big branches of machine learning happening at Meta1.

When we started building LLMs, we mostly focused on them knowing things.

The capabilities in a large language model emerge, mysteriously, from the training data.

There has been a lot of drama1 about distillation: how (closed) frontier models are being used by other labs to boost their own performance on particularly hard tasks.

A thing you can do is take the most performance and correctness sensitive part of your stack and just ask a chatbot to write it for you.

SIMT offered a fantastic bargain.