Learn AI
Triton Kernel Mastery: From tl.load to Flash Attention
Write custom GPU kernels in Triton from first principles — the block programming model, masking, fused softmax, autotuning, blocked matmul, and Flash Attention with online softmax, then PyTorch and torch.compile integration. Interview-framed with quizzes and spot-the-bug challenges.
GPU Performance Engineering: CUDA & SYCL
How GPUs actually execute code and how to make kernels fast — the execution model, memory hierarchy, coalescing, bank conflicts, occupancy, the roofline model, warp divergence and reductions, and a profiler-driven optimization workflow. The interview backbone for GPU/perf roles.
GEMM from Scratch: How a Matmul Reaches 90% of Peak
Optimize a GPU matrix multiply step by step — naive kernel, coalescing, shared-memory tiling, register blocking, vectorized loads, tensor cores, and closing the gap to cuBLAS. The canonical 'walk me through optimizing a kernel' interview exercise, with the real speedup numbers.
vLLM & Distributed Kernels: Interview Prep
A FAANG interview bootcamp for LLM inference & distributed GPU systems — PagedAttention, continuous batching, and the vLLM scheduler, then collective communication, ring vs tree all-reduce, NCCL topology, and custom fused all-reduce kernels. Conceptual Q&A, spot-the-bug, and system-design rubrics.
PyTorch Interview Prep: Amateur to Expert
An 8-week FAANG interview bootcamp for PyTorch — tensors and autograd through torch.compile, FSDP2, and ML system design. Conceptual Q&A, live-coding solutions, spot-the-bug challenges, and design rubrics, all on verified PyTorch 2.x APIs.
How vLLM Scales Across GPUs
A WebGL walkthrough of vLLM V1's four parallelism dimensions — tensor, pipeline, Wide EP, and disaggregated serving — with animated all-reduce, pipeline bubbles, expert dispatch, and KV transfer, each grounded in the merged PRs behind it.
vLLM Optimization Deep Dive
Master vLLM's optimization internals through Triton kernel implementations.
NKI Kernel Programming
Program AWS Trainium from first tile to Flash Attention using NKI — the CUDA of Neuron hardware.
The Attention Family: A Visual Guide
A 3Blue1Brown-style WebGL journey through attention and its variants — from Q/K/V and the T×T score matrix to MQA, GQA, MLA, sparse, linear, and FlashAttention, each animated to show how it beats the O(T²) cost.
Want more content on a specific topic?
Suggest topics for GPU kernels, ML optimization, or systems programming
Get in Touch