🔢 Tensors – Maitreya

Strides, views, broadcasting, and the memory model underneath every tensor op.

How the GPU Actually Runs Your Tensor Code

tensors

gpu

Two expressions can compute the same result and differ enormously in cost — sometimes the slower one is the one doing less arithmetic. How synchronization, launch overhead, memory traffic, compute intensity, allocation, and dynamic shapes compete to be your bottleneck, and how to measure which one actually dominates.

Jul 20, 2026

Deep dive

Ten Tensor Problems Where Shape Is Not Enough

tensors

drill

A closing self-check on the tensor fundamentals: ten small problems where an answer can be semantically wrong despite a plausible shape — or mathematically right but unusable because of layout, numerics, device placement, or synchronization cost.

Jul 20, 2026

Note

Numerical Stability: The Window, and the Precision You Lose Inside It

tensors

Floating-point arithmetic is not real arithmetic — it can only represent numbers inside a finite window, and outside it, exp overflows to infinity and tiny probabilities collapse to exactly zero. Many of the most common stability techniques share one move: rewrite the computation so dangerous intermediates stay representable. Cancellation then adds a second problem — preserving significant digits even when every intermediate is comfortably in range. Here are both, and the six places they matter — including the one that returns negative squared distances, turns them into NaN, and corrupts distance magnitudes without raising anything.

Jul 19, 2026

Deep dive

Selecting and Placing: gather, scatter, and the Indexing Model

tensors

gather is the operation everyone re-derives from the docs every single time. Here is the one sentence that makes it stop being mysterious, the intuition that it’s a one-hot matmul without the matrix, and the mutation rule that separates a copy from a view — where writing to the wrong one silently does nothing.

Jul 19, 2026

Deep dive

Broadcasting, Reductions, and Einstein Summation: The Algebra of Axes

fundamentals

tensors

PyTorch broadcasting will line up two tensors of different shapes and compute something — and when the shapes mean different things, that something is a silently wrong answer with a valid gradient. Here is the rule, a worked derivation of exactly what wrong objective you get, and the notation that makes axis intent reviewable.

Jul 16, 2026

Deep dive

A Tensor Is a Flat Buffer and a Promise

fundamentals

tensors

How PyTorch maps a logical index like x[b, h, s, d] to an address in flat memory — and why views, reshapes, and packed layouts can hand you the right shape with the wrong values. The one idea that turns tensor gymnastics from memorized incantations into arithmetic you can derive.

Jul 14, 2026

Deep dive