📱 Small Models – Maitreya

Sub-billion parameter models - where the budget actually goes, and what it costs to deploy them.

A Small Model Is a Budget, Not a Parameter Count

llm-systems

small-models

architecture

At small scale, parameter count hides the decisions that determine the model you actually deploy: vocabulary, KV state, depth, width, and active versus resident weights. This post derives those budgets from scratch, measures the tokenizer trade-off, and benchmarks equal-size models with a real KV-cached decode path.

Oct 22, 2026

Deep dive