simons blog

Radix Top-K is an algorithm for finding the top-k elements in an array without sorting the full array.

In CuTeDSL, it is often convenient to use StaticPersistentTileScheduler when writing persistent kernels.

SBO and LBO are essential to write correct Kernels on B200 GPUs that leverage Tensor Cores.

In this short note we will briefly derive a helpful identity to speedup GDN prefill algorithm.

Chunkwise Gated Delta Rule is important when performing operations such as prefilling or training with the recently popular Gated Delta Attention formulation.

In this blogpost we will briefly explain Gated Delta Net decoding.

In this blogpost we will discuss the kernel of grouped blockscaled GEMM on B200.

This is the next blogpost where I aim to explain grouped blockscaled GEMM for B200 GPUs in a top down manner.

This is the first blogpost on a series which will be about Grouped Blockscaled GEMM on Blackwell of which the code can be found here.

Warp specialisation is an optimisation that splits the GEMM mainloop into two parts: We have one warp that does the TMA (i.e. copy tiles to SMEM) and one warp that does the MMA (i.e. multiply together these tiles).