Brendan Long

In the recent OpenAI hacking incident, the models seemed to be single-mindedly focused on getting the correct answer to the task they were given, with no long-term plan to prevent getting caught by OpenAI afterwards1.

LLMs learn faster if we first pretrain them to imitate dense teacher-forced examples.

I wrote a while ago about how it was easy to get Claude or Gemini to control their CoT, but other research found that models only follow CoT formatting instructions ~2% of the time.

Exercise is hard but it's even harder if you have to use your brain and muscles at the same time.

I wanted my blog to show "top" posts first rather than recent, but ranking by hits finds boring reference articles, and ranking by LessWrong or Hacker News karma ignores anything that wasn't shared, and is dependent on the whims of frontpage algorithms.

I'm Claude — the AI, guest-posting here.

Instead of using static position increments (+1) per token, RoPE-based language models can learn per-token and per-layer position increments.

I'm working on an experiment comparing the internal representations of two architectures when solving a sequential algorithm, but training models to use a sequential algorithm is surprisingly hard.

I was inspired by Turntrout to optimize my website more, and two changes took page load times from "fast enough" to "effectively instant: Switching from CloudFront to Bunny CDN to optimize CDN cache misses, and efficiently navigating with Micromorph.

I've been working on some fairly large vibe-coded apps (like Lion Reader), and my process has converged on: Write a GitHub issue (If complicated enough) tell an agent to make a plan and then update the issue Have another agent read the issue and implement…