nathan.rs

The typical language model is autoregressive (AR): it predicts one token at a time, left to right, each token conditioned on the ones before it.

A while back I wrote about language modeling without neural networks, where I generated Shakespeare with an unbounded n-gram model: no weights, no training, just counting.

Diffuji is an instant camera built at TreeHacks 2026.

Generating Shakespeare has become the “Hello World” of language models.1 Recently, I’ve been messing with alternative language models and came across unbounded n-gram models.

In this post, I run small experiments showing that diffusion language models generate code (and other structured text) at a faster rate.

For Cal Hacks 2025, a few friends and I built Curserve, a fast and scalable server-side engine for agentic coding, which ended up placing for one of the sponsor prizes.

A while back, Google DeepMind unveiled Gemini Diffusion, an experimental language model that generates text using diffusion.

Here are some notes I wrote over this topic.

A few weeks back, I implemented GPT-2 using WebGL and shaders (Github Repo) which made the front page of Hacker News.

These are my notes from Mark Maxwell’s courses — Probability I and Introduction to Mathematical Statistics — and his textbook, Probability & Statistics with Applications, Second Edition.

Lately, I’ve been coming across many blogs that have weird font-size rendering issues for code blocks on iOS.

For a while, I wanted to build a complete autograd engine.

Lexical Analysis and ASTs# Recently I was going through Thorsten Ball’s “Writing An Interpreter in Go”.

These are my notes from Qiang Liu’s Machine Learning II course at UT Austin, cleaned up and stitched into a single story (with much help from Claude).

These are my notes from Eunsol Choi’s NLP class at UT Austin, cleaned up and stitched together into a single story (with much help from Claude).

These are my notes from working through Gilbert Strang’s Introduction to Linear Algebra.

Why Rust for Front-End Development# I’ve been using React and Next.js for front-end development ever since high school, it was one of the first few things I learned when it came to programming.

A short review of Calculus 1, 2, and 3, based on Calculus: Early Transcendentals (Eighth Edition).

This is a poem I wrote my senior year of high school in AP Literature.