Hoanh Le

Overview In this post, I’ll iteratively optimize an implementation of the attention backward kernel written in CUDA for the 5090.

Overview Welcome to my 3D model of Aalto University, where I spent my master’s studies.but mostly hanging out and tinkering on different projects with my friends. :) This project is made possible by recent advances in Gaussian Splatting.

Attention is all you need Since their introduction in 2017, transformers (Vaswani et al., 2017) have revolutionised NLP, and now applications over all domains in Deep Learning.