Notes on math and machine learning.

Family life has decreased my writing cadence — sorry about that.

https://arxiv.org/html/2507.16003v1 I liked this paper, I think it gives a nice way to think about how self-attention (and equivalent constructions) operate.

In this post, we will first summarize what we discussed in part 1 and part 2, then add a few extra thoughts.

In the previous post we talked about creating attracting manifolds and how to get the corresponding vector field given a manifold.

TL;DR: This is a math post (and part 1 of a theme, here's part 2).

I discovered this nice set of notes by Paige Bright, Alan Edelman, and Steven G.

I was listening to StarTalk and Neil deGrasse Tyson told the story of the discovery of Neptune, which first happened mathematically and then observationally!

TL;DR: My notes on the “Greedy Coordinate Gradient”, first read from Zou, et al, 20231, but slightly abstracted.

TL;DR This paper studies a subset of possible transformer models called “compact transformers” (CT).

First of all, happy new year to all!