Jittered Sample Weights

You can't have your cake and eat it too. In graphics the cake is a small buffer: when you add up samples for antialiasing, path tracing or light map baking, you want to accumulate them in as few bits as you can get away with. A smaller buffer takes less memory, and it is faster as well, because much of the cost of accumulating is the time spent reading and writing that memory.

Eating it is the banding. Sum a lot of samples into a low bit depth buffer and you run into it quickly: every add is rounded to what the buffer can store, similar samples all round the same way, and the error grows with each one until it shows as bands.

The obvious answer is a buffer with more bits, which gives up the memory and the speed. But if you only need a bit more headroom, there is a cheaper first step: give each sample a slightly different weight. The weights still sum to the same total, but each add now lands at a different point between two representable values, so some round up and some round down, and the errors mostly cancel instead of piling up. You get small, fast and free of bands at once.

I first used this in 2016 for supersampling in 8-bit buffers. This week it turned out to work in half floats too, for light map baking. The two cases need different weights, because fixed point and floating point round differently. In both, the same buffer takes four to eight times more samples before it bands.

Fixed point: SSAA in 8 bits#

SSAARenderPass in Three.js renders the scene $n$ times with sub-pixel camera offsets and adds each render to the output with a weight of $1/n$. In 2016 that output was an 8-bit buffer.

Take a pixel in the interior of a surface, where all $n$ samples have the same value $v$. Each add contributes $255v/n$ steps, rounded to a whole number of steps. The result is

$$n \cdot \operatorname{round}\!\left(\frac{255\,v}{n}\right),$$

which can only be a multiple of $n$ steps. With 32 samples, an 8-bit buffer is left with nine distinct levels, and everything darker than 6% gray sums to black.

An artificial case: every sample of a pixel is identical, as in the interior of a surface. The GIF uses a fixed palette of 256 grays, so the only bands are the accumulated ones.

The fix, which I worked out with Daniel Sturk and added in mrdoob/three.js#8930, spreads the weights evenly over a small range around $1/n$:

const baseSampleWeight = 1.0 / jitterOffsets.length;
const roundingRange = 1 / 32;

for ( let i = 0; i < jitterOffsets.length; i ++ ) {

	// uniformly distributed over -0.5..0.5, so the weights still sum to 1
	const uniformCenteredDistribution = - 0.5 + ( i + 0.5 ) / jitterOffsets.length;
	const sampleWeight = baseSampleWeight + roundingRange * uniformCenteredDistribution;

	// render sample i and add it to the output with this weight

}

A range of $1/32$ moves a white sample's contribution across eight steps, and a sample of value $v$ across $8v$ steps. Anything brighter than $1/8$ is spread over at least one full step, which is what makes the rounding even. It is still in the pass today as the unbiased option.

Mean absolute error over values from 0 to 1. Rounding the exact average once gives 0.25 steps, the best an 8-bit buffer can do.

With equal weights the error doubles each time the sample count doubles. With jittered weights it stays near one step up to 64 samples. Past 64, a $1/32$ range around $1/n$ reaches below zero, the negative weights are clamped, and the curve rejoins the other one.

Floating point: light map baking in half floats#

The Three.js LightMapper bakes a light map by summing irradiance samples in a half float target, with the sample count in alpha, and divides by the count at the end. Johnathon Selstad found that it banded at 256 samples and moved the sums to 32-bit floats in mrdoob/three.js#35431. That is the right fix where the GPU can blend float targets. Where it cannot, I suggested the weight jitter for the half float fallback, and he added and measured it in the same PR.

Floating point changes two things.

The rounding step grows with the sum. A half float has 11 significant bits. After $i$ samples of value $v$ the sum is about $iv$, and its rounding step is between $iv/2048$ and $iv/1024$. A fixed jitter range cannot keep up with that. The range has to grow with the sample index, so sample $i$ gets a weight of $1 \pm i/2048$, which spreads its contribution over one full step of the sum at that point.

The weights have to be accumulated too. The baker does not know each texel's sample count in advance, because coverage varies. Multiplying the whole output, color and alpha, by the weight makes alpha accumulate the sum of the weights, and the existing divide at the end stays correct.

_sampleWeight( index ) {

	if ( this._floatSums ) return 1;

	return ( 1 + ( 2 * halton( index + 1, 5 ) - 1 ) * Math.min( index / 2048, 1 ) ) * HALF_FLOAT_WEIGHT;

}

// in the material
material.outputNode = vec4( irradiance, 1 ).mul( this._weight );

HALF_FLOAT_WEIGHT is $1/64$. Scaling by a power of two does not change the rounding, and it keeps the sums of bright texels below 65504, the largest half float.

The same artificial case in half floats: identical samples of a smooth irradiance field. The error follows contours of equal irradiance, which is why it reads as bands. At 4096 samples the equal-weight count has stuck at 2048, because 2048 + 1 rounds back to 2048, and the result is far too bright.

Mean relative error over values from 0.01 to 4.

Jittered weights cut the error from 1.3% to 0.20% at 256 samples and from 5.6% to 0.62% at 1024. The simulation only has rounding error. Johnathon's numbers are from real bakes of the Cornell Box at 512², measured against a path traced reference, so they include the bias of the bake itself, about 0.43%:

The first row is worth a look. With 2 rays per sample the samples are noisy, and equal weights are already fine. Noise in the samples does the same job as jitter in the weights. Banding is a problem of clean samples, so it gets worse as the renderer gets better.

What it costs#

Memory: nothing. This is the reason to use it. Float sums take 16 bytes per texel instead of 8, so a 2048² light map accumulates in 64 MB instead of 32 MB. Jittered weights moved the usable limit of the half float baker from 256 samples to 1024 in the same 32 MB. In 8 bits, 32 jittered samples have the same error as 4 equal ones, without a wider buffer.

Code: one uniform and one multiply. The weight is the same for every pixel of a pass. There is no noise texture and no per-pixel hash, and the result is deterministic.

A little variance. Unequal weights make a Monte Carlo average slightly less efficient. With weights of $1 \pm i/2048$ the variance rises by 0.2% at 256 samples, 2.8% at 1024 and 11% at 2048.

A ceiling. Weights cannot go below zero. The 8-bit version runs out at 64 samples and the half float version at 2048, where the weights span 0 to 2. Jitter gets a buffer four to eight times further than equal weights. It does not replace a wider buffer when one is available: float sums stayed at 0.43% at every sample count above.

The recipe#

• Fixed point. Spread the weights evenly over a fixed range around $1/n$. The range should move the darkest value you care about across one rounding step: $1/32$ covers values above $1/8$ in 8 bits. It works up to $n = 2/\text{range}$ samples.
• Floating point. Accumulate the weighted sample and the weight, and divide at the end. Let the range grow with the sample index: $1 \pm i/2048$ for half floats, from any well-distributed sequence. It works up to about 1024 samples.
• Either way, check whether your samples are already noisy. If they are, you may not need it.

The figures in this essay come from a small simulation of both accumulators, not from a GPU.