The Noise Schedule Decides Where Your Timesteps Land
Both previous posts took the linear schedule from the DDPM paper without comment. It decides how a fixed budget of timesteps is distributed across signal-to-noise ratios — and the linear and cosine schedules distribute it very differently, which is one reason cosine was introduced and often behaves differently from the original linear schedule.
generative
diffusion
Published
July 31, 2026
The first post opened with torch.linspace(1e-4, 0.02, 1000) and moved on. That line is the noise schedule, it was chosen by hand in the DDPM paper, and of everything in these three posts it’s the part with the least justification and the most consequence.
The question a schedule answers is: at timestep \(t\), how much of the original signal is left? That’s \(\bar\alpha_t\), and the useful way to look at it is as a signal-to-noise ratio:
A schedule decides how densely a fixed timestep budget samples the noise-level axis. Very-low-SNR steps are not useless — they are ordinary training examples. P²-weighting interprets that region as emphasising coarse, global structure, but the threshold count below does not itself measure usefulness, gradient quality, or learned detail. What it measures is where the timesteps land.
TL;DR — On a 1000-step budget the linear schedule puts about a third of its steps below SNR 0.01, crossing at around t = 674; the cosine schedule puts about 6% there, crossing near t = 936. That’s a redistribution of capacity, not a recovery of otherwise-useless steps. It’s one reason cosine was introduced — though it didn’t universally replace linear, and modern systems use linear, scaled-linear, cosine, learned log-SNR and sigma-based parameterisations depending on the setup. A second, separate issue: linear leaves a small but non-zero amount of signal at \(t=T\), so training and sampling start from slightly different distributions.
Two schedules
import torch, mathtorch.manual_seed(0)T =1000def linear_abar(T): betas = torch.linspace(1e-4, 0.02, T)return torch.cumprod(1.0- betas, dim=0)def cosine_abar(T, s=0.008, max_beta=0.999):"""Nichol & Dhariwal. Defines the target ᾱ directly, then caps the implied β. The cap is not optional. At t = T the cosine argument is exactly π/2, so the raw target ᾱ(T) is mathematically **zero** — which makes the implied β(T) = 1, hence α(T) = 0, and DDPM's reverse step divides by √α. The reference implementation caps β at 0.999 for exactly this reason. Built in float64 deliberately: in float32 the endpoint evaluates to a rounding residue near 1e-15 rather than to zero, and that residue is an artefact of the dtype (float64 gives ~1e-33), not a property of the schedule.""" t = torch.arange(T +1, dtype=torch.float64) / T f = torch.cos((t + s) / (1+ s) * math.pi /2) **2 target = f / f[0] betas = (1- target[1:] / target[:-1]).clamp(max=max_beta)return torch.cumprod(1- betas, dim=0).float()ab_lin, ab_cos = linear_abar(T), cosine_abar(T)# This is the CAPPED cosine — see the docstring. Without the cap the schedule's# terminal α is zero and the DDPM reverse step is singular.snr_lin, snr_cos = ab_lin / (1- ab_lin), ab_cos / (1- ab_cos)print(f"{'t':>6}{'ᾱ linear':>12}{'ᾱ cosine':>12}")for t in [0, 200, 400, 600, 800, 999]:print(f"{t:>6}{ab_lin[t]:>12.4f}{ab_cos[t]:>12.4f}")
The shape of the difference is already visible: the linear schedule destroys signal fast and early, the cosine one holds onto it and collapses late.
Measuring the allocation
Set a threshold defining a very-low-SNR regime and count how many timesteps fall below it:
def schedule_report(name, abar, threshold=0.01): snr = abar / (1- abar) low_snr = (snr < threshold) first_low_snr =int(low_snr.float().argmax()) if low_snr.any() else Tprint(f"{name:>8}: ᾱ(T) = {abar[-1]:.2e} "f"SNR < {threshold} from t = {first_low_snr:<4} "f"({low_snr.float().mean():.1%} of all steps)")schedule_report("linear", ab_lin)schedule_report("cosine", ab_cos)
linear: ᾱ(T) = 4.04e-05 SNR < 0.01 from t = 674 (32.6% of all steps)
cosine: ᾱ(T) = 2.43e-09 SNR < 0.01 from t = 936 (6.4% of all steps)
A third of the linear schedule’s budget sits below SNR 0.01. What that means for training is an empirical question rather than something the count settles. P²-weighting offers the interpretation usually cited here: it groups very low SNR with coarse, global structure, intermediate SNR with perceptually rich content, and high SNR with cleanup and fine detail. Read through that lens, the threshold count measures how much timestep resolution each schedule assigns to the coarse-structure regime — it does not by itself measure usefulness or gradient quality.
Two related lines of work reweight the loss across noise levels rather than moving the steps, and they are motivated differently: P² prioritises the perceptually useful regimes, while Min-SNR treats timesteps as competing optimisation tasks whose gradients conflict. Both are consequences of noise levels being non-uniformly informative; neither is the same argument.
The threshold is a judgement call, so it’s worth checking the conclusion isn’t an artefact of picking 0.01:
print(f"{'threshold':>10}{'linear':>10}{'cosine':>10}")for thr in [0.05, 0.01, 0.001]: l = ((ab_lin/(1-ab_lin)) < thr).float().mean() c = ((ab_cos/(1-ab_cos)) < thr).float().mean()print(f"{thr:>10}{l:>9.1%}{c:>9.1%}")
The gap holds across two orders of magnitude of threshold. It’s a property of the schedules, not of where the line is drawn.
The other end, which matters for a different reason
There’s a second, subtler issue at the top of the range — whether the schedule actually reaches pure noise by \(t=T\):
print(f"{'':>8}{'ᾱ(T) [power]':>16}{'√ᾱ(T) [amplitude]':>22}")for name, ab in [("linear", ab_lin), ("cosine", ab_cos)]:print(f"{name:>8}{ab[-1]:>16.2e}{ab[-1].sqrt():>22.2e}")print(f"\nlinear at t=T: {ab_lin[-1]*100:.4f}% of the variance, "f"{ab_lin[-1].sqrt()*100:.4f}% of the amplitude")
ᾱ(T) [power] √ᾱ(T) [amplitude]
linear 4.04e-05 6.35e-03
cosine 2.43e-09 4.93e-05
linear at t=T: 0.0040% of the variance, 0.6353% of the amplitude
Both columns are the same fact stated two ways, and it’s worth being precise about which: ᾱ is a variance coefficient, √ᾱ is the amplitude scaling on \(x_0\). Linear retains 0.004% of the variance, equivalently 0.635% of the amplitude.
Why it matters: sampling starts from \(x_T \sim \mathcal{N}(0, I)\), i.e. from the assumption that ᾱ(T) is zero. If it isn’t quite, you train on one distribution and sample from another. I haven’t measured what this costs on MNIST, so I won’t claim it’s negligible there. Where it has been shown to matter is for the image statistics that survive at very low SNR — overall brightness and other low-frequency statistics — and some modern training configurations enforce zero terminal SNR for that reason. Scheduler APIs generally expose it as an option rather than applying it by default. The capped cosine used here leaves about 5e-5 of the amplitude at t=T — a direct consequence of the β cap, which is what stops the schedule from reaching an exactly-zero terminal α.
What this changes in practice
The architecture and the update equations don’t change. What does change is the whole schedule tuple — q_sample and DDIM read only ᾱ, but DDPM’s reverse step also reads β, α and the posterior variance, and replacing only abar would leave the sampler inconsistent with the process it’s inverting:
# What actually has to change. `q_sample` and DDIM read only ᾱ; DDPM's reverse step# also reads β, α and the posterior variance, so all of them must be re-derived# from the SAME schedule or the sampler and the training distribution disagree.def schedule_from_abar(target_abar, max_beta=0.999):"""Derive a self-consistent schedule. Note the recomputation of ᾱ after clamping: if the clamp bites, the requested ᾱ and the achievable one differ, and returning the requested one would hand back a tuple whose parts describe different processes.""" prev_t = torch.cat([target_abar.new_ones(1), target_abar[:-1]]) betas = (1- target_abar / prev_t).clamp(max=max_beta) alphas =1- betas abar = torch.cumprod(alphas, dim=0) # re-derived, not the input prev = torch.cat([abar.new_ones(1), abar[:-1]])returndict(abar=abar, betas=betas, alphas=alphas, posterior_var=betas * (1- prev) / (1- abar))sched = schedule_from_abar(ab_cos)print("re-derived:", ", ".join(sched))print(f"β range: {sched['betas'].min():.2e} … {sched['betas'].max():.3f}")
There’s a distinction here worth stating carefully, because the previous posts depend on it. Changing the training noise parameterisation — reassigning what ᾱ a given integer timestep means — generally requires retraining, or a correct remapping of the model’s noise-level conditioning: the network learned a denoiser for the levels it saw, and querying it at others is querying it off-distribution.
Changing the inference solver, or selecting a subset of the already-trained noise levels, does not. That’s precisely what the first post does when it runs 50-step DDIM over a model trained with 1000 steps — same trained levels, fewer of them visited, different update rule. Diffusion libraries expose exactly this: compatible inference schedulers are swappable — Diffusers publishes a compatibility set per pipeline rather than treating all schedulers as interchangeable — and inference timesteps or sigmas are configurable, all without touching the weights.
The interaction with sampler step count is real but weaker than it first appears. Strided samplers select timesteps from the schedule, so a schedule that allocates a third of its range to very low SNR will, under uniform striding, put a comparable share of your 50 steps there. Whether that costs you depends on the striding rule — implementations that space steps uniformly in log-SNR rather than in t largely sidestep it, which is itself an argument that the schedule and the sampler’s step selection should be designed together.
What to carry away
The schedule allocates a fixed budget across noise levels. SNR = ᾱ/(1−ᾱ) is the axis it allocates along, and it’s more informative on a log scale than a linear one.
Linear puts about a third of its steps below SNR 0.01; cosine about 6%. The gap holds across two orders of magnitude of threshold, so it’s a property of the schedules rather than of where the line is drawn. Those steps remain valid training examples; the schedule is simply assigning a large share of its discrete resolution to the very-low-SNR regime.
The implied β must be capped. The raw cosine target reaches exactly zero at t = T, so β = 1 and α = 0, and DDPM’s reverse step divides by √α. Capping β at 0.999 avoids the singularity and leaves a small nonzero terminal ᾱ.
Terminal SNR is a separate issue. Linear retains 0.004% of the variance at t=T, so training and sampling start from slightly different distributions — which matters most for the image statistics that survive at very low SNR.
Distinguish the two kinds of change. Redefining what noise level a timestep means needs retraining or a remapping of the conditioning. Choosing a different solver, or a subset or spacing of the levels the model already knows, does not — that’s what DDIM is.
If one sentence survives: the schedule doesn’t create a useful/useless boundary — it determines how densely a finite set of timesteps samples the SNR axis.
Where this goes next
Three posts in, and everything has run in pixel space on 28×28 images. Pixel-space diffusion does scale — Imagen and the cascaded models are pixel-space — but the cost grows quickly with resolution, which is why the dominant approach runs the whole process in a compressed latent instead. That raises the question worth a post of its own: what the autoencoder throws away, and why the diffusion model turns out not to need it.