Ankit Maloo

Two days on Opus 4.8, then 34 hours on Fable 5.

Summary: Today, llms are trained in a multi step process post SFT.

Some problems don’t get solved in your head, they get solved in someone else’s while you watch or read their solution.

Some elements here build upon the RL env post.

Introduction I occasionally like to reverse engineer computer games by playing against inbuilt AI and trying different tactics to find loopholes.

AI trends move fast.

Richard Sutton wrote about the ‘Verification Principle’ in 2001: An AI system can create and maintain knowledge only to the extent that it can verify that knowledge itself.

someday i will make a more thorough and better reasoned post around it but at this point these are just notes somewhat structured The current discourse around LLM memory feels fundamentally misguided.

From Scott Alexander’s 2015 post Douglas Adams once said there was a theory that if anyone ever understood the Universe, it would disappear and be replaced by something even more incomprehensible.

The Race for AI Progress In 2019, Richard Sutton, wrote his groundbreaking essay titled ‘The Bitter Lesson’.