kalomaze's kalomazing blog

Intro If you're active at all when it comes to language model research these days, you almost certainly have an implicit mental model for what kinds of training have been useful for capability improvement in the post-o1 era.

I think there is a serious anti-pattern that sometimes shows up in papers doing online reinforcement learning at scale.

Hi!

Author's Note This is an experimental piece - I recorded an hour long voice memo while on a walk, talking to myself out loud, stream of consciousness style.

Introduction Many GRPO reproductions for LLM reinforcement learning available online lack useful intuitions or recommendations regarding hyperparameters and reward shaping.

DeepSeek.

Note: This is a repost of an older rentry page: https://rentry.org/corruption_model_writeup I've been building a custom model specifically tailored to take input text that has been "corrupted" in specific sections.