Skip to content
Ben is currently available for contract work for 3D & web solutions — reach out.

Back to Blog Listing

How Games Denoise Reflections, and What Three.js Does Instead

Shipped reflection denoisers agree on eight steps, and the ones built for screen-space reflections do it in three passes or fewer. Three.js has two denoisers: the one its SSR example uses, which hides its problems by darkening, blurring and filling, and an SVGF-based one that was added and then pulled from that example. This is the recipe, the comparison, and the three steps still missing.

Ben Houston • • 22 min read

Stochastic reflections trace one ray per pixel, so the raw result is noise. A denoiser turns it into an image. How Games Do Screen-Space Reflections, and What Three.js Does Instead names one thing the Three.js denoisers get wrong: they fill misses. This post covers that and the rest.

It does four things. It sets out what shipped reflection denoisers have in common, with the sources. It measures three Three.js denoisers against that list: the pair the SSR example uses today, the SVGF-based DenoiseNode that replaced it for five days, and my changes on top of that node. It says what is still missing. And it says what DenoiseNode can stop doing, because one denoiser is enough and it should be the cheaper one.

Which denoiser to copy#

There are two weight classes.

The heavy class is built for ray tracing. NVIDIA's NRD runs eight passes for specular: tile classification, hit distance reconstruction, a pre-pass, temporal accumulation, a history fix, a blur, a post-blur and temporal stabilization. SVGF, the paper most of this field descends from, runs five filter iterations over a 65 pixel footprint. As published it took most of a 60 frames per second budget at 1080p, so nobody ships it that way. What ships is its idea: let the measured noise of each pixel decide how far the filter reaches.

The light class is built for screen-space reflections. Frostbite's stochastic SSR reuses rays between pixels inside the tracer, and after that its denoiser is one temporal filter. AMD's FidelityFX reflection denoiser is three passes: reproject, one spatial filter, one temporal resolve.

The light class is the one to copy. A browser cannot afford eight passes for reflections alone, and screen-space rays are cheaper and better behaved than path-traced ones. What makes the comparison useful is that both classes agree on almost everything except how many passes they spend.

The recipe#

Three requirements on the input, and eight steps.

Input: light, not shaded color. The denoiser filters incoming light and the BRDF is applied afterwards. NRD: "all materials must be de-modulated before denoising". Frostbite gives the practical reason: "It is much better to multiply by [FG] after the temporal filter, so that the changes in it do not affect the neighborhood clamping." AMD's sample multiplies by the pre-integrated BRDF in its apply pass, after the denoiser.

Input: a rule for misses. A ray that leaves the screen needs its answer from somewhere else. Engines supply it either before the denoiser or after it, and both are accepted. The next section compares them. What does not work is writing black and leaving the denoiser to guess.

Input: noise that is independent between pixels. NRD is designed around white noise as its baseline, and says blue noise improves on it. Every filter below averages neighbours on the assumption that their errors differ. If neighbours share an error, averaging keeps it.

1. Look for the history in two places. A reflection does not move with the surface it is seen in. It moves with the thing reflected. Frostbite: "we calculate the average depth of the reflected objects, and reproject using that". AMD and NRD compute both positions, the one from surface motion and the one from the hit point, read the history at each, and choose per pixel by how well the normal and roughness found there match. Rough surfaces use surface motion. Mirrors use the hit point.

2. Reject history from another surface, and count what is left. Each history tap is tested against the current pixel's plane and normal. Every engine keeps a per-pixel count of accumulated frames, and the blend weight for the new frame is one over that count. A pixel that was just uncovered starts at one.

3. Cap the count, lower for smooth surfaces. NRD defaults to 30 frames. Lumen defaults to 32. AMD caps at max(8, maxSamples * (1 - exp(-roughness * 100))), so a near-mirror keeps eight frames and a rough surface keeps the full length. A mirror's reflection changes completely with a small camera move, so a long history there is lag.

4. Clamp the history against something less noisy than one frame. History that survives reprojection can still be stale. Temporal anti-aliasing clamps it to the range of the current 3×3 neighbourhood. That works when the current frame is clean. A one-ray frame is not, so its 3×3 range is either enormous or wrong. Frostbite clamps after the ray-reuse resolve, when each pixel already holds four samples. NRD keeps a second, short history of six frames and clamps the long one to that.

5. Size the spatial filter by how noisy the pixel is. This is the idea in SVGF's name: variance-guided. Track the variance of each pixel's luminance over time, and let neighbours differ by a few standard deviations before the filter stops. AMD's spatial pass is a "variance guided edge aware filter" of "15 samples in a 7x7 region". NRD's ReBLUR gets to the same place without variance: its blur radius is scaled by 1 / (1 + k * N) of the frame count N. Both mean the filter is wide when the pixel knows little and narrow when it has converged.

6. Stop at edges of the geometry and of the lobe. Depth or plane distance, normal, roughness and hit distance. The normal tolerance follows the width of the specular lobe, so a mirror only accepts neighbours facing the same way and a rough surface accepts more.

7. Give new pixels a wider filter. A pixel with two frames of history has no usable variance. SVGF estimates it from the neighbourhood "where our temporal history is limited (<4 frames after a disocclusion)". NRD runs a separate history fix pass for the first three frames.

8. Skip what does not need it. AMD does not trace above a roughness threshold, 0.2 in its sample, and writes the environment map there. Lumen stops at 0.4. And at the other end: "Every pixel will use denoising unless the surface is perfectly smooth (mirror-like)". A mirror ray has no noise to remove.

What the heavy class adds on top is more passes, a firefly filter, sharper history resampling and a final stabilization pass. None of it changes the list.

Filling misses before the denoiser, or after#

A material's reflection comes from several sources: SSR, reflection probes and the environment map. Engines differ on whether those are combined before the denoiser runs or after it.

Before. Each ray that missed is given a fallback radiance inside the tracer, so the denoiser receives a finished signal. AMD's tracer samples the environment map and blends it in by the confidence of the hit. Lumen continues the ray with a slower trace, "a more reliable method if no hit is found", and denoises what comes back.

After. The tracer outputs light and a coverage, the denoiser filters both, and the lighting code fills what is left from probes and the environment. This is Frostbite's pseudocode, HDRP's reflection hierarchy and Filament's one line of shader, all quoted in the series opener.

Fill before denoisingFill after denoising
Used byAMD FidelityFX SSSR, LumenFrostbite, Unity HDRP, Filament, Godot
What the denoiser getsA finished signal with no gapsLight and coverage
Where the other sources are readIn the tracer, once per rayIn the material, once per pixel
StrengthThe denoiser needs no notion of coverageProbes and the environment are already prefiltered, so they skip the denoiser and stay sharp and free of lag
CostThe tracer has to reach every fallback source, and clean sources pick up the denoiser's blur and lagThe denoiser has to carry coverage through every stage
FitsA fallback that is itself traced and noisy, or a single fallback sourceSeveral prefiltered sources that the material already reads

This series fills after. SSR is denoised on its own, with coverage, and the material runs the cascade. Three.js is a forward renderer: the probes, the lightmap and the environment are chosen and read in the material, so filling in the tracer would mean a second copy of that logic in a post-process. SSRNode's private environment map is what that looks like. And the sources SSR falls back to are clean. A prefiltered probe has no noise to remove, and sending it through a temporal filter only makes it lag.

A miss is not a hole. A ray that leaves the screen has given an answer: the screen does not know what is in this direction, so ask the next source. A hole is a pixel with no data, where the best guess is whatever its neighbours have. A denoiser that cannot tell them apart treats every miss as a hole.

Shipped denoisers do fill holes. AMD can trace one ray for a 2×2 quad and copy it to the other three pixels. NRD reconstructs checkerboarded input, and marks a pixel that traced nothing with a hit distance of zero so that it can be rebuilt from the 3×3 around it. Those pixels traced no ray. Engines are less strict about misses at a small scale. Godot's SSR gives a missed sample a reduced weight in its blur, so on a smooth surface a hit spreads a little way into the misses beside it. None of them does what the Three.js pair does, which is to replace a missed pixel with its neighbours' color and report it as covered.

The rule in this series. The tracer writes radiance and coverage, 1 for a hit and 0 for a miss, with radiance premultiplied by coverage. Both are filtered with the same weights, so a miss counts as a sample that covered nothing. The statistics that steer the filter, the variance and the clamp, are taken over covered samples only. After denoising, a pixel's coverage is the fraction of its rays that hit, and its radiance divided by its coverage is the average of what those rays saw.

HDRP gets to the same place by another route. Its spatial pass averages only the neighbours that hit and takes coverage from the pixel's own ray, and its temporal pass then averages color and coverage together.

The three Three.js denoisers#

The pair. TemporalReprojectNode and RecurrentDenoiseNode, from #33843. The first reprojects the previous output. The second filters it with eight taps and blends in the new frame, and its output becomes the next frame's history. This is what webgpu_postprocessing_ssr_denoise runs today.

DenoiseNode. Johnathon Selstad's SVGF-based node, from #35318 and #35450. One node: a temporal pass with a fast history and a variance estimate, then two filter passes of nine taps each. It replaced the pair in the SSR example on November 6 and was taken back out on November 11. It is still in the repository and still denoises SSGI and lightmaps.

DenoiseNode with coverage. My ssr-coverage branch: the same node, with coverage carried through every stage, and several of NRD's refinements added as options.

The pairDenoiseNodeWith coverage
Light, not shaded colorNo. The tracer applies a BRDF estimate, and the filter has an albedo edge stop to cope with itNot its concern, but the example fed it shaded colorYes. The tracer outputs radiance and the material applies its BRDF
A rule for missesBlack means no sample, and is filled from neighboursThe tracer's environment sample. A coverage option exists, but its statistics count a miss as blackCoverage. Statistics, clamps and edge stops use covered samples only
Independent noiseR² sequence, correlated between neighboursRequired and documentedHashed per-pixel noise
1. Two history lookupsYes, blended by confidenceOne lookup, at a position blended by roughnessThe same
2. Reject and countYesYes, and it follows TRAA's jitterThe same
3. Capped, lower when smoothFixed at 16Fixed at 8Fixed at 20
4. Clamp to a cleaner signalNo. Clamps to a 3×3 of single raysYes. A four-frame fast historyYes, on radiance, with NRD's thresholds as options
5. Filter sized by noiseRadius shrinks with the frame count. The luminance stop is a fixed thresholdYes. Variance, as in SVGFYes, on radiance
6. Geometry and lobe edge stopsYes, with a kernel aligned to the lobeYes, with a screen-aligned kernelThe same
7. Wider filter for new pixelsIn part. Brightness-weighted, which darkensIn part. A 3×3 estimate where SVGF uses 7×7The same
8. Skip mirrors and rough pixelsNoNoNo

The pair meets three steps outright, two in part, and none of the input requirements. DenoiseNode meets four outright and one in part. With coverage it meets the same five and all three inputs.

The pair#

One part of it is worth keeping: its two-lookup history, with a confidence for each lookup, is the best reprojection of the three. That is a function to move, not a reason to keep the node.

The rest is where it falls short.

It has no idea how noisy a pixel is. The luminance edge stop is a fixed number: a neighbour is rejected when its luminance differs by more than a set amount. That amount is right for one exposure. Make the scene ten times brighter and the filter stops ten times sooner. And the comparison is between two single rays, so it is noise against noise.

It leans on brightness to hide outliers. For the first five frames, each tap is weighted by the inverse square of its luminance. The temporal blend weights history and new frame by inverse luminance again. The history clamp compresses bright values before measuring them. Each one suppresses fireflies by trusting dark samples more than bright ones, and each one darkens the result. Selstad measured the pair at 11% darker than a converged reference.

It clamps against noise. The history is clipped to the variance of a 3×3 window of raw single rays. That is the temporal anti-aliasing clamp applied to a signal it was not designed for.

Its noise is correlated, and the recurrence hides it. SSRNode drives the pair with an R² sequence in which diagonal neighbours share a sample dimension 94% of the time. That is a band, not noise. The pair removes it by feeding its blurred output back into itself every frame until the structure is gone, at 2.6 times the error of a filter that did not have to.

And it fills misses. SSRNode writes black for a miss, and both nodes read black as absent. In RecurrentDenoiseNode, a pixel whose own ray came back black is dropped from its own average:

If( raw.rgb.length().lessThan( 0.0001 ), () => {

	denoisedRaw.assign( vec3( 0 ) );
	totalWeightRaw.assign( 0 );

} );

Its value then comes entirely from its neighbours. TemporalReprojectNode makes the same assumption from the other side: a black sample does not count as a frame, so the history keeps showing last frame's reflection. A reflection of something that really is black gets the same treatment, and is brightened by what surrounds it.

Filling does three kinds of damage.

  • It invents reflections. At the edge of a reflected object, half the rays hit the object and half go past it. Filling gives all of those pixels the object's color. The reflection grows a soft border that is not in the scene.
  • It hides the tracer. The SSR animation example finds 46% of the hits it should. The denoiser fills the gaps from the rays that did land, and the picture looks fine. That is how Three.js came to ship approximate SSR.
  • It blocks every other source. A filled pixel has an answer, so reflection probes and the environment map are only asked where a whole neighbourhood missed.
The raw screen-space reflection buffer of the Three.js SSR animation example: three reflective spheres on a smooth floor. The floor's reflection of the spheres and the wall behind them is a sparse scatter of lit pixels on black, and the tops of the spheres, which reflect the sky, are solid black

One frame of raw SSR from the animation example, before the pair fills it. The black on the spheres is correct: those rays reflect the sky. The black gaps on the floor are rays the tracer should have landed.

Four rows of the same strip of pixels across the edge of a reflected object: the raw rays, the filled result with the object's color spread past its edge, the coverage curve falling from one to zero, and the resolved result blending the object into the environment map.

The same row of pixels, filled as the pair does and resolved by coverage.

Measured against a many-frame reference in the SSR example, the pair's error is 3.05% standing still and DenoiseNode's is 1.98%. On the same machine the two cost the same to within 7%.

DenoiseNode#

It gets the core right. It tracks variance per pixel and sizes its edge stop in standard deviations. It clamps the long history to a short one. It keeps its own copy of last frame's depth and normals, so it depends on nothing outside itself. In screen space it never fills: there is no test for black anywhere in that path.

So why did it come out of the SSR example? Two reasons, and neither was the denoiser.

The first was banding. Soon after the merge the example showed diagonal bands. They were the R² correlation, now visible because nothing was blurring it away. Michael Herzog put it exactly: the old denoiser "filtered in a way that broke up the R² noise's spatial structure. The new SVGF doesn't do that anymore so it needs a different noise."

The second was flicker. With the noise replaced, the bands went and the reflections still shimmered where the pair's had been steady. I traced that to the anti-lag. Where the floor reflects the edge of a dark object against a bright sky, a pixel's ray either hits the object or misses to the sky. The four-frame history swings between the two, disagrees with the long history, and the anti-lag reads that as a change in the scene and throws the history away. Those pixels reset every 37 to 67 frames, depending on the history length, and never finish converging.

That is not a tuning problem. It is a hit and a miss being averaged as if they were two samples of one signal. An honest denoiser was put behind a tracer with no coverage, and it showed what the tracer was doing. The series opener predicted this failure for any fix made on its own: every individual change is correct and looks like a regression.

With coverage#

The ssr-coverage branch changes what DenoiseNode is given and how it reads it. One change matters.

A miss carries coverage, not color. The tracer outputs radiance and coverage, with no BRDF and with per-pixel hashed noise. In the denoiser, the neighbourhood statistics, the fast history, the clamp and the variance are computed on the radiance of covered samples, weighted by coverage, and only in frames that have data. A hit next to a miss is no longer a luminance edge, and the swing that reset the history is gone from the signal.

The branch also raises the history from 8 frames to 20 and turns the anti-lag off for reflections.

It adds six more options, taken from NRD: variance clipping around the fast history, a relaxed anti-lag threshold, a lobe footprint that shrinks as the history grows, an edge stop that tightens the same way, Catmull-Rom resampling of the history, and a firefly clamp. All six are off in every example on the branch. They were experiments, and the section on cost below argues for deleting most of them.

Does that close the gap?#

Most of it. The inputs are right, the noise estimate is right, the clamp has a sound reference, and a miss is a miss. Three steps are still open.

Step 1: two lookups. DenoiseNode blends the surface position and the hit-point position into one history position, by roughness, and reads once. Halfway between two places is neither of them. It also uses the pixel's own raw hit distance, which on a rough surface is as noisy as the color. The pair already does this properly. The fix is to move its two-lookup history, and its 3×3 average of hit distances, into DenoiseNode.

Step 3: a history cap that follows roughness. One line, from AMD. Without it, smooth surfaces either lag or the whole image runs a short history.

Step 8: skip mirrors and rough pixels. Every shipped denoiser spends nothing on a mirror pixel and nothing on a pixel too rough to trace. Ours runs every pass on both.

What it costs, and what to cut#

DenoiseNode has a reputation for being heavy. Its documentation says SVGF "is designed for desktop-class hardware", and adding it to the SSGI ball pool example took an M2 Pro from 60 frames per second to 46. But that example had no denoiser before. In the SSR example, where it replaced the pair, the cost did not change. The pair is not the cheap option. It is the same price for a worse result.

So the question is not which denoiser is faster. It is what DenoiseNode does that reflections do not need. For reflections it runs three full-screen passes:

PassTexture reads per pixelWrites
TemporalAbout 30Four targets: history, fast history, depth guide, normal guide
Filter, firstAbout 40Two targets
Filter, secondAbout 40One target

It keeps about 96 bytes per pixel in render targets, which is 200 MB at 1080p. Two thirds of that is the history, kept twice so that one copy can be read while the other is written. A quarter is the two intermediate targets between the filter passes.

Here is what to cut, largest first.

1. Pixels with nothing to denoise. Today the only pixels skipped are the background. A matte wall goes through all three passes. So does a mirror, and so does every pixel whose rays all missed. This is step 8 of the recipe. In a scene where half the screen is matte, it halves the cost. It needs one roughness test and one coverage test at the top of each pass.

2. The second filter pass, for reflections. Two passes of nine taps exist because the input is one ray per pixel. Exact Screen-Space Reflections on a Budget puts ray reuse in front of the denoiser: each pixel is resolved from four neighbouring rays, weighted by its own BRDF. That is a spatial filter already, and a better one, because it knows the lobe. Frostbite runs nothing after it but the temporal pass. AMD runs one spatial pass. With reuse in front, reflections need one pass, sized by variance to clean up what is left. That saves a third of the reads and both intermediate targets.

The pass count is already a setting, iterations, and it should stay a setting. SSGI and anything else without a reuse stage in front still want two. The change is to the reflection examples, which set it to one once ray reuse is on.

3. Reading the same texel twice. Roughness is read separately from the normal it is stored beside. Hit distance is read twice in the temporal pass. In the filter, each tap reads the normal guide once for the normal and again for the hit distance. None of this changes the image.

4. The firefly clamp. A firefly is one ray that found something very bright. The denoiser is the wrong place to deal with it, because by then it is one sample and the only tools left are to clamp it, which loses energy, or to down-weight it, which is how the pair came to be 11% dark. The tracer can stop it being produced. Ray reuse averages four rays and normalizes by the sum of their weights, so no single ray can dominate. Reading the hit color from the mip that matches the cone's footprint means a small bright object arrives already averaged over the area the ray stands for. Both are steps in the tracer post. With them in place the clamp has nothing to do.

5. Catmull-Rom history. Five more reads per pixel to keep a long history sharp. NRD needs it: its histories default to 30 frames and are built for sharp ray-traced mirrors. Ours is 20 frames of a glossy reflection, followed by a spatial filter. The blur it prevents cannot be seen.

6. The remaining options. Variance clipping, the shrinking footprint and the tightening edge stop each restate something the variance estimate already does. They cost little at run time and a good deal in settings nobody can tune. The anti-lag stays in the node, because SSGI uses it, and stays off for reflections.

The 'quality' preset, with its extra pass and its plane fit at every tap, is for lightmaps, where it runs once.

The end state#

Put together, the denoiser this series is building has two stages and a history.

The structure of the proposed reflection denoiser. Radiance, coverage and hit distance from the screen-space reflection pass join into one line that enters the denoiser on the left and passes through a temporal stage and a spatial stage. One line leaves on the right and splits into denoised radiance and coverage for the material. The G-buffer, holding depth, normal, roughness and velocity, feeds both stages from above, and the temporal stage reads and writes a history that holds the running average of past frames.

The denoiser at the end of this work.

Ray reuse is not in the diagram because it is not in the denoiser. It belongs to SSRNode, and what arrives on the left has already been through it. Nothing here is new. Steps 1 to 4 of the recipe are the temporal stage, steps 5 to 7 are the spatial stage, and step 8 is a test at the top of both. Each already exists in Three.js or on a branch, in one node or another.

The order#

  1. Noise. Independent per-pixel noise in SSRNode. Nothing downstream can be judged without it.
  2. Coverage and radiance from the tracer, as in Exact Screen-Space Reflections on a Budget.
  3. DenoiseNode carries coverage, and takes its statistics from covered samples.
  4. Two-lookup history and the roughness cap. The lookup code comes from the pair.
  5. Remove TemporalReprojectNode and RecurrentDenoiseNode. Nothing is left in them that DenoiseNode does not do better.
  6. Classification: no denoising on mirrors, matte pixels or pixels with no coverage.
  7. Ray reuse on in SSRNode, with iterations set to one for reflections.
  8. Delete the firefly clamp, Catmull-Rom and the redundant options.

The pair should go. It reaches a steady image by darkening, blurring and filling until the problems upstream cannot be seen, it costs as much as the denoiser that does none of those things, and keeping both means every SSR change has to be tuned twice. DenoiseNode is the right foundation. The work left is to finish its reprojection, and then to make it smaller.