Deschide articolul original într-un tab nou Schimbarea acestei opțiuni se aplică imediat și reîncarcă pagina.
LECTURA
Înapoi la articole →

ML Perf Notes

danielvegamyhre.github.io/feed.xml
Vizitează site-ul

Cel mai recent articol

Ultima verificare

Urmăritori 0 urmăritori
Articole: 4

  1. MXFP8 GEMM: Up to 99% of cuBLAS performance using CUDA + PTX

    (ML Perf Notes)

    I recently did a deep-dive on writing GEMM kernels with just CUDA + PTX for Ampere, Hopper, and Blackwell GPUs, culminating in a MXFP8 GEMM kernel which achieves up to 99% of cuBLAS (torch.

  2. Debugging deadlocks in warp-specialized GEMM kernels with CUDA-GDB

    (ML Perf Notes)

    While writing warp specialized GEMMs for Blackwell with CUDA + PTX, I’ve noticed there are some great resources on efficient kernel designs for this architecture, but a lack of resources on debugging the often cryptic or even completely opaque CUDA errors…

  3. An illustrated deep-dive into how the compute and comms in TP+SP are overlapped using Async TP

    (ML Perf Notes)

    In a previous post we did a deep-dive into Megatron-style tensor parallelism.

  4. An illustrated deep-dive into Megatron-style tensor parallelism

    (ML Perf Notes)

    In this post, I attempt to provide a detailed walkthrough of Megatron-style tensor parallelism, with diagrams to help make the concepts and mathematics more digestible.