Aman's blog

I got into contributing to open-source (for me that was llama.cpp) at a uniquely opportune time.

I just came back from a vacation to Japan with my wife and 10-month old.

The MTP PR got merged into llama.cpp a few days back.

This post about about a PR to llama.cpp which adds support to prefetch weights, overlapping compute for the current layer with the weight loading for the next layer.

Autoresearch is all the rage, it's like people like never heard of while loop before.

I recently needed to delete objects in a LIFO manner and employed an LLM to rubber-duck about what would be the best container.

Note: this is a copy of the discussion here LLM inference is divided into 2 phases: prompt-processing (PP) and token-generation (TG).

There are a lot of resources online about writing a fast GEMM, and they all get complicated really fast, and by the time you reach the tensor core section you need to keep a lot in context to understand.

I have a git repo where I put in my life goals.

Contributing to open-source projects is a goal of many programmers.