The Quest for AAA Quality in Three.js

Three.js is having a moment. A year ago it was downloaded about 3 million times a week from npm. This month it passed 34 million.

Weekly npm downloads of three, as posted by @threejs.

Every November, Codevember fills timelines with Three.js pieces that are simply gorgeous: a sculpted object turning slowly under a studio environment map, with perfect reflections and soft shadows.

But walk into a scene and the illusion breaks. Put the camera inside a room, a corridor or a courtyard, and a Three.js scene does not yet look like a modern game. The light does not bounce off the walls. The floor reflects the sky instead of the room it is in. Corners are not quite dark enough, and the shiny objects in them reflect light they should not see.

Those are the two gaps: global illumination and reflections. Good models and materials matter too, but that is a separate story. This post is about rendering technique, and the good news is that for rendering technique this is a solved problem. We do not need to invent anything. We need to look carefully at what AAA games already ship, pick out what works on the web, and build it into Three.js as one coherent pipeline.

That is the quest: make Three.js competitive with Unreal Engine for indoor scenes, at a fraction of the size, on most of the devices people actually own. By Unreal I mean Unreal with baked lighting, not Lumen. This post is the plan, and a progress report.

It should not cost you a second toolchain either. The baking runs in the browser: load a model, unwrap it, bake it, and see the bounced light in seconds, with no round trip through Blender or a game engine. That part already works today.

"We" in this post means the contributors doing this work, mainly me and zalo, together with the Three.js maintainers who review it and shape it.

None of this takes anything away from what Three.js does well today. A single object under an environment map will render exactly as it does now, at the same cost. The pipeline is opt-in, and the large majority of it lives in the addons under examples/; changes to the core in src/ are kept to the minimum necessary. The aim is to widen what Three.js can do, from the product viewer all the way to an indoor first-person game, without giving up either end.

Where we are heading: SSR, box-projected reflection probes and the environment map combined in one Three.js corridor (PR #35046).

The games do the research; Three.js does the distillation#

Three.js has always adopted techniques from AAA games rather than invented them, and for good reason. Game studios target known hardware: a console, or a desktop GPU above some minimum spec. They ship engines measured in hundreds of megabytes and games measured in tens of gigabytes. They can spend a year tuning one renderer for one platform.

Three.js has none of those luxuries:

• It has to run everywhere from one code base, on the majority of phones and desktop computers people own, from a gaming PC to a mid-range phone. This is one of mrdoob's core requirements for the project.
• The code has to be small. It ships to every visitor of every site that uses it.
• The assets have to be small. Nobody waits for a 2 GB download before a web page appears.
• There is no hardware ray tracing. WebGPU does not expose it, and will not on most devices for years.

For this pipeline I am proposing two more constraints.

WebGPU first. The pipeline is built for WebGPURenderer only. The maintainers have made clear that WebGPURenderer is the future of Three.js and that the dedicated WebGLRenderer is in maintenance mode, so nothing here is backported to it. WebGPURenderer also has a WebGL 2 backend, and the pipeline supports that backend wherever possible. WebGPU is available on about 87% of devices today, according to Web3D Survey, a site I built to track exactly this, and that number is rising quickly.

No exotic features. Within WebGPU, the pipeline should use only what is built in, or optional features that are close to universal: roughly 97% or more of WebGPU devices on Web3D Survey. Whether to go a little lower depends on how much a feature is worth, but the default is to run with no optional features enabled at all. Where something below that bar is worth having, it is an upgrade with a fallback, never a requirement: RG16 snorm normals fall back to RG16F, and BC6H texture compression, which is mostly a desktop feature, falls back to rg11b10ufloat.

Because of its nature, Three.js tends not to push the state of the art: it has to run nearly everywhere, where a AAA game can limit itself to a few consoles and high-end PCs. What it can do, and what it is good at, is consolidation: take the techniques AAA games have proven, find the subset that runs on roughly 90% of consumer devices, and make that subset small, fast and easy to use. That is a different and very useful kind of work. The games did the research; we do the distillation.

A few terms first#

Lighting papers use several names for the same things. These are the ones this post uses:

• Direct lighting: light arriving straight from a light source.
• Indirect diffuse (irradiance): light bounced off other surfaces, arriving from all directions.
• Indirect specular (radiance): bounced light seen in the mirror direction; reflections.
• Ambient occlusion (AO): how much nearby geometry blocks indirect light.
• Specular occlusion (SO): the same, for the reflection direction.
• Coverage: how much of a pixel a technique could answer, from 0 to 1.
• BRDF: the material's shading function: how a surface turns incoming light into the light you see.
• G-buffer: per-pixel images of the visible surface (depth, normal, roughness) that screen-space effects read.
• SSR: screen-space reflections, found by tracing rays through the depth buffer.
• SSGI: screen-space global illumination; the same tracing, gathering bounced diffuse light instead of reflections.
• TRAA: temporal anti-aliasing; each frame is blended with the reprojected frames before it.
• PMREM: an environment map pre-blurred for every roughness level, which is how Three.js lights from a cubemap.

What AAA games do#

If you are a graphics developer, reading about how games render is one of the great pleasures of the job. I started where I always start, with the granddaddy: Lagarde and de Rousiers' Moving Frostbite to Physically Based Rendering. It has been around the longest, and it is the paper we used to bring physically based materials to Three.js years ago, which was one of my previous quests to improve it. This time I am leaning on it again, for its sections on reflection probes and on combining indirect lighting.

Conference talks and papers matter too, and GDC and SIGGRAPH Advances are where the engine teams publish. But my main resource has been something else: frame breakdowns of shipped games written by people outside the teams, who capture a frame in a GPU debugger and walk through every pass. They are some of the best technical writing on the internet, and they show what actually ships, not just what was presented. A few blogs that do this regularly:

• Adrian Courrèges' graphics studies, including DOOM (2016), plus a compilation of everyone else's.
• Behind the Pretty Frames, long breakdowns of games such as Death Stranding and Elden Ring.
• Emilio López's The Code Corsair.
• Thomas Poulet's frame analyses.
• Simon Coenen's DOOM Eternal graphics study.

The most useful games to study are the ones that look great and do it the way a web renderer can: without real-time ray tracing. They split indoor lighting into two problems, global illumination and reflections, and their answers to each are remarkably consistent.

Global illumination#

For global illumination, these are the games that show the recipe most clearly.

They share one recipe:

1. Bake indirect diffuse into lightmaps for everything that does not move. Do it offline, with a slow, high-quality path tracer. Store irradiance only, without the surface color, and multiply the color in at runtime. Global illumination is low-frequency, so lightmaps can be low-resolution: the texture detail comes from the material.
2. Light dynamic objects with probes: a grid or volume of points that store incoming light in every direction, sampled and interpolated wherever a character or prop happens to be.
3. Keep the key lights real-time. They can still move, flicker and cast sharp shadows and highlights, while their bounce is baked.
4. Add screen-space ambient occlusion for the small-scale contact shadows a lightmap is too coarse to hold, and for where dynamic objects meet static ones.

Put together, each mesh first takes one of two paths, depending on what kind of mesh it is. A static mesh with a lightmap reads the lightmap. Everything else reads the probe grid, but a probe grid only covers the volume it was placed in. Outside that volume, the environment map serves as one global irradiance probe. The grid's coverage fades to zero near the volume's edges and the environment map fills in the rest, 1 − coverage, so an object walking out of the grid changes lighting smoothly instead of popping. Ambient occlusion then darkens the result either way. Some engines put a screen-space stage in front of all of this; that is the dashed box at the top, and it is discussed below.

Indirect diffuse: optional SSGI, then the lightmap, or the probe grid falling through to the environment map, then ambient occlusion.

The trade-off is that baked bounce light is static. If you move a wall, or switch off a light, its bounce does not update. For most indoor scenes, from architecture to retail to most games, that is an excellent trade, and it is the one these games made.

What about SSGI? Screen-space global illumination is the diffuse counterpart of screen-space reflections: it traces rays through the depth buffer and gathers the bounced light it finds on screen. None of the games above is known to use it, but general-purpose engines with dynamic GI do:

The baked games do without it for two reasons. A lightmap already holds the static bounce, so SSGI added on top counts it twice. And SSGI loses light the moment its source leaves the screen, which is far more noticeable in diffuse lighting than in a reflection. Placed in front of the lightmap and probe grid in a coverage cascade, though, it replaces the baked bounce instead of adding to it, as HDRP's does, and contributes what a bake cannot know: light bounced off moving objects and from moving lights. It is an upgrade worth having on top of the recipe, not part of it.

Reflections#

For reflections, the shipped-game consensus is just as strong.

Every one of these is a cascade. Screen-space reflections are tried first, because they are sharp and correct, but they can only reflect what is already on screen. Wherever SSR has no answer, the renderer falls back to local reflection probes: cubemaps captured inside each room. Beyond the probes is the sky, the distant environment, which always has an answer.

The key idea that makes a cascade work is coverage. Each stage outputs not just a color but how much of the pixel it could answer, from 0 to 1. The next stage fills in the rest, 1 − coverage. SSR gives high coverage on a floor reflecting a wall that is on screen, low coverage near the screen edge, and none for a reflection pointing out of the window. The probes fill the gap.

The reflection cascade: each stage answers what it can, and the next fills in the rest.

Diffuse uses a shallower version of the same idea. As in the global illumination recipe above, lightmapped meshes read the lightmap, and everything else falls through from the probe grid to the environment map. SSGI, where it is used, goes in front of both, exactly as SSR goes in front of the reflection probes.

Building the pipeline in Three.js#

Putting the two surveys together gives one frame that looks like this:

One frame of the pipeline. The TRAA output feeds the next frame's SSR and SSGI.

None of these pieces is new; every one is in the games above. What is new is this particular combination in Three.js, chosen because it looks excellent, runs in real time on ordinary hardware, and stays small in both code and download size.

Where things stand#

This work started in September. Two months in, here is how each stage compares with what the games require. Each row is explained in its own section below:

The biggest remaining gaps are in the SSR and BRDF rows: Hi-Z tracing and coverage for SSR (and, for the optional SSGI stage, the same coverage and previous-frame work), and above all the pluggable indirect lighting that ties both cascades into the BRDF. Everything else is in Three.js, in review or drafted.

It has moved this fast because of the Three.js maintainers. Michael Herzog (@Mugen87) has reviewed most of these PRs, and more than once has reworked one into a better design that fits how Three.js as a whole is built, adjusting the renderer's internals to make room for it. mrdoob has expressed strong support for many of them. I am very lucky to have both.

So far, though, these pieces have been proposed and reviewed one at a time, each on its own merits. Nobody has been asked to sign up to the whole. This post is my attempt to put them together into one coherent picture, so that each PR can be judged for where it fits as well as for what it does.

The pipeline, stage by stage#

The rest of this post walks through the frame one stage at a time. For each, it asks three questions: what do the games require, what does Three.js already have, and what is missing. The answers to the third question are where the work is.

The G-buffer#

What the games require. SSR, GTAO and the denoisers all run in screen space before lighting, so they need a per-pixel description of the visible surface: a G-buffer. A deferred engine stores everything lighting will need, including base color and emissive, because its lighting pass reads only the G-buffer. A forward renderer like Three.js has an advantage here: the beauty pass shades each material directly, so the G-buffer only needs what the screen-space passes read.

Base color, emissive and the material AO map are not on the list: the beauty pass reads them straight from the material.

What Three.js has. WebGPURenderer already renders a G-buffer prepass with multiple render targets, and the SSR examples use it today. Normals are packed into RGB8 with roughness in alpha, base color with metalness in a second RGBA8 target, and velocity in a third:

const scenePass = pass( scene, camera );
scenePass.setMRT( mrt( {
	output: output,
	diffuseColor: vec4( diffuseColor.rgb, materialMetalness ),
	normal: vec4( packNormalToRGB( normalView ).rgb, materialRoughness ),
	velocity: velocity
} ) );

So the mechanism exists. The gaps are the layout and the normal precision.

G-buffer structure and memory#

A layout that covers the table above, in three render targets plus depth:

Sixteen bytes per pixel is lean by AAA standards. Deferred engines commonly spend 20–28, because they also store base color, emissive and baked lighting.

The G-buffer is not the only screen-sized memory, though. The rest of the frame adds a half-float beauty target (8 B), TRAA history and output (16 B), the previous depth for reprojection (4 B), a Hi-Z pyramid (about 5 B with its mips), and half-resolution SSR, denoiser history and GTAO buffers (about 7 B together). That is roughly 56 bytes per pixel for the whole screen-space pipeline:

Does it fit? Yes. A desktop GPU with 8 GB or more barely notices 200–465 MB. A phone with 6–12 GB of shared memory can afford about 100 MB, and phones already render below native resolution, which is what makes the phone row realistic. The tighter budget on mobile is bandwidth rather than capacity: reading and writing all of this every frame moves around 15 GB/s on that phone at 60 fps, against 50 GB/s or more of memory bandwidth on a recent mid-range device. That is why SSR, its denoiser and GTAO run at half resolution, and it is comfortably within reach. Half resolution has one requirement of its own: the results must be upsampled with a depth- and normal-aware filter, or reflections and occlusion bleed across the edges of objects. I have drafted that filter as jointBilateralUpsample(); more on it under occlusion below.

Normal precision#

The one field Three.js stores too coarsely today is the normal. SSR is unforgiving of normal error: a ray reflected off a slightly wrong normal lands somewhere visibly wrong at the end of a long trace. With RGB8 normals, smooth curved surfaces snap to a limited set of quantized directions, and their reflections show banding and clumping.

AAA G-buffers solve this with octahedral encoding: fold the sphere of directions onto an octahedron, unfold it into a square, and store two coordinates. It is the most accurate of the encodings surveyed by Cigolle et al. for its size. We have added packNormalToOctahedral() and unpackOctahedralToNormal() to Three.js in PR #35480:

That is fifteen times more accurate with half floats, and over a hundred times with snorm, for the same four bytes. RG16 snorm is the target, with RG16F as the fallback.

Reflections traced from RGB8 normals (left) show banding; octahedral RG16F normals (right) are smooth.

Angular error of the stored normal: RGB8 (left) and octahedral RG16F (right), at the same scale.

Global illumination#

What the games require. From the survey:

• An offline solver that bakes bounce light into lightmaps without leaking light through walls.
• Lightmap UVs for every static mesh, with padding so charts do not bleed into each other, and seams that do not show.
• A compact way to ship lightmaps, which are HDR, and a compact way to hold them in GPU memory.
• A probe volume to light dynamic objects, ideally one that also does not leak.

What Three.js had. When I started on this in September, the runtime side had been there for years. Every standard material takes a lightMap, read from a second UV set:

material.lightMap = bakedTexture;
material.lightMap.channel = 1; // the lightmap UV set

The lightmap example shows it, but with lighting baked ahead of time in another tool and shipped as ordinary textures. The only way to bake inside Three.js was zalo's ProgressiveLightMap (example), which accumulates jittered shadow maps into a lightmap over many frames. It produces lovely soft shadows, but no bounced light, and it reuses each mesh's existing UVs, packing whole objects into the atlas instead of unwrapping them.

For dynamic objects, LightProbeGrid places a grid of probes through a scene, each storing incoming light as L2 spherical harmonics, and lights each object from the probes around it (example). It worked well; the limit was how long a capture took.

The gaps, and how we filled them. Measured against the requirements, almost the whole authoring side was missing.

• No unwrapper. Most models do not come with a second, non-overlapping UV set. A real-time UV unwrapper now generates a lightmap atlas for any mesh, in the browser.
• No bounce-light baker. The lightmap baker is a framework for baking indirect diffuse into those UVs, with pluggable solvers.
• No high-quality solver. zalo's ray radiosity is the solver that makes the baker worth using. It traces against three-mesh-bvh, so light cannot pass through walls, and bakes a Cornell box in under a second. His denoiser turns a fast, noisy bake into a clean one.
• Probe capture speed. Batching probe captures made LightProbeGrid four times faster, and later changes doubled that again: it now bakes about eight times faster than it did at the start of September, and supports multi-bounce light.

The unwrapper's lightmap atlas for the Damaged Helmet, one color per chart.

zalo's courtyard at 64 samples per texel, raw and denoised.

Sponza lit by LightProbeGrid.

Together, the unwrapper, the baker and ray radiosity are a complete in-browser baking pipeline: load a model, unwrap it, bake it, and see the result in seconds.

Remaining gaps.

• Seams. Where a surface is split across two UV charts, the two sides are baked separately and can disagree, which shows as a visible line. zalo's seam stitching and gutter filling fixes this and is in review.
• Leaking probes. A probe on one side of a wall can light an object on the other, so a dark room glows with the light of the bright room next door. DDGI solves this by storing, with each probe, the distance to the nearest surface in every direction, and ignoring probes that cannot see the point being lit. DDGI-style visibility weighting for LightProbeGrid works as a draft PR, but it is not yet cheap enough per pixel. Getting that cost down is the remaining work.
• Shipping lightmaps in glTF. There is no standard glTF extension for lightmaps. We have proposed KHR_lightmap, currently a draft.
• Compact HDR files. Lightmaps need HDR, which rules out PNG, and EXR or HDR files are large. UltraHDR JPEG and JPEG XL are both HDR and very small over the wire. UltraHDR works in every browser today, because it is an ordinary JPEG plus a gain map, so it is the near-term choice. JPEG XL compresses better and is the long-term one, but browser support is still limited. Three.js already loads UltraHDR, and I recently added HDR JPEG XL loading with exactly this in mind, for both lightmaps and environment maps, so it is ready when browsers are. The next step is glTF extensions for those two texture formats.
• Compact GPU memory. A small JPEG still decodes to a large texture: a 4096² lightmap is 134 MB as half-float. Encoding it to BC6H, the GPU's compressed HDR format, cuts that to 17 MB. It can be done once at load with a real-time compute encoder (Narkowicz, Real-Time BC6H Compression on GPU; there is a WGSL port), writing the blocks to a buffer and copying them into the texture. Where BC6H is not available, rg11b10ufloat halves the memory instead.

With those in place, Three.js has a complete baked GI pipeline that runs in the browser, ships quickly and uses little memory on the device that displays it.

Ambient and specular occlusion#

What the games require. Lightmaps are too coarse for contact shadows, and they know nothing about dynamic objects. Games add screen-space occlusion on top, and Activision's GTAO is the standard. It also produces a bent normal, the average open direction, which gives specular occlusion (GTSO): a reflection pointing into a fold of the surface is occluded, one pointing out of the opening is not.

What occlusion multiplies matters as much as how it is computed:

• AO multiplies all indirect diffuse: lightmaps, probes and the environment, combined with the material's AO map as min( materialAO, gtao ).
• GTSO multiplies the probe and environment part of indirect specular.
• Neither touches SSR hits. An SSR hit is something the ray actually found, so it is already correctly occluded.
• Neither touches direct light, which has shadows.

Applying AO to lightmapped surfaces darkens them slightly twice, since the bake already includes static self-occlusion. We do it anyway, because dynamic objects still need to occlude static ones.

What Three.js has. GTAO is already there, and it already feeds the lighting rather than being multiplied over the final image. The AO example renders a prepass, runs GTAO on it, and hands the result to the beauty pass through a lighting context:

const aoPass = ao( prePassDepth, prePassNormal, camera );
scenePass.contextNode = builtinAOContext( aoPass.getTextureNode().sample( screenUV ).r );

This is exactly the pattern the rest of the pipeline needs, and it matters again below.

The gaps. There are two.

• Specular occlusion. Today GTAO outputs a single visibility value, and the physical lighting model turns it into specular occlusion with a heuristic based on the view angle and roughness. It does not know which directions are blocked, so metal keeps reflecting the sky inside folds. Bent normals and GTSO are up as a draft PR.
• Edge-aware upsampling. Half-resolution GTAO upsampled with a bilinear filter leaves dark outlines around objects, because occlusion from the surface behind bleeds across each silhouette. jointBilateralUpsample() upsamples using the full-resolution depth and normals as guides (Kopf et al. 2007), so the result stops at the edges. It is drafted and not yet submitted, with the AO example as its first user; half-resolution SSR needs the same filter.

With GTSO (left), folds and cavities stop reflecting the sky. With a scalar specular occlusion (right), they still glow.

Reflections: screen-space reflections#

What the games require. From DOOM, Frostbite and the others, SSR in a cascade has to:

1. Trace efficiently over any distance, using a Hi-Z depth pyramid.
2. Output coverage, so the probes can fill what it misses.
3. Trace the previous frame's lit image, so its result can feed the current frame's lighting.
4. Denoise spatially and temporally, so one ray per pixel is enough.

What Three.js had. In September, Three.js's SSR was built for a different job: adding reflections on top of a finished image, in a compact scene. It took few samples, so rough surfaces were noisy. It was composited over the image, so it did not go through the BRDF. And its rays stopped at a fixed distance, so reflections of anything far away were missing.

Much of that has changed since. SSRNode now importance-samples the full GGX lobe with correct Fresnel, validates and refines its hits, and uses Houston Stratified Noise, also known as HHW (Hierarchical Hashed Weyl) noise, in place of a blue noise texture. Its result is reprojected and accumulated over time, and fed back so reflections show reflections.

What it does not yet do is fit into a cascade. Here is how the SSR example combines it with the scene:

const litColor = scenePassColor.rgb.add( denoisedReflections.rgb );

The beauty pass renders first, SSR traces that same image, and the reflections are added on top. Each requirement above is a gap from there.

Hi-Z tracing. SSRNode currently marches the depth buffer in up to 64 steps out to a maxDistance, with a thickness tolerance for deciding a hit. Long rays are expensive, and both values need tuning per scene. A Hi-Z pyramid is a mip chain where each texel stores the minimum depth of the four below it. The ray takes large steps through empty space at coarse levels and fine steps only near surfaces, so cost grows with the log of the ray's length. Every ray can then go as far as the screen allows, with no maximum distance to tune. Storing both the minimum and maximum depth in the pyramid also helps the thickness test: a single depth layer cannot tell whether a ray passed behind an object or into it, and the extra bound narrows that ambiguity. The technique is from Uludag's Hi-Z Screen-Space Cone-Traced Reflections in GPU Pro 5, building on the screen-space ray marching of McGuire and Mara. Frostbite (for smooth surfaces), AMD's FidelityFX SSSR and Killing Floor 2 all use it. This one is not started yet.

Hi-Z ray marching: large steps across empty space, small steps near the surface.

Coverage. Today, a ray that misses samples an environment map itself, inside SSR: an equirectangular texture passed in as environmentNode, which cannot be the PMREM-filtered scene.environment, let alone a local probe. In a cascade, SSR should instead output its reflection and its coverage, lowered near the screen edge, at grazing angles, behind thin geometry and on very rough surfaces, and leave the rest to the probes. We have prototyped this; it lands together with the BRDF integration below.

Tracing the previous frame. SSR cannot be part of the lighting while it reads the lighting's output. DOOM's answer is to trace the current depth buffer but read colors from the previous frame: reproject each hit point into last frame's TRAA output and sample it there. Last frame's image already contained last frame's reflections, so a mirror facing a mirror gains a bounce each frame. Previous-frame tracing is in review.

Two rules keep this loop healthy. First, the image SSR reads must be linear HDR radiance, taken after TRAA but before tone mapping, bloom and other post-processing; otherwise reflections pick up tone-mapped colors and bloom halos. Second, the fed-back radiance must be clamped. Each frame's reflections become the next frame's input, so a single firefly or a slight energy gain in the denoiser compounds frame after frame instead of fading away.

Reflections only. With previous-frame feedback, reflections appear inside reflections.

Denoising. Rough reflections need more than one ray's worth of information per pixel, and Three.js does not yet have a spatio-temporal denoiser designed for SSR. Frostbite's is the model: reuse rays from neighbouring pixels, weighted by each pixel's own BRDF, then accumulate over time, reprojecting history by the reflected hit point's motion rather than the surface's, since reflections move differently from the surfaces that show them. zalo is working on this. His SVGF-based DenoiseNode has merged, but the SSR examples have gone back to the previous denoiser while it is reworked for reflections. The next attempt will be judged with a harness, described below.

Screen-space global illumination#

SSGI is optional in this pipeline. The baked recipe does not need it, but it adds bounce light that a bake cannot know about, and it reuses almost everything SSR needs.

What the games require. Engines that use SSGI as part of their lighting, rather than as an effect laid over the image, treat it the way they treat SSR. Unity HDRP is the plainest example: where SSGI is enabled it replaces the lightmap and light probe data, and where a ray finds nothing on screen it falls back to reflection probes and the sky. In a cascade, that means SSGI has to:

1. Output coverage: the fraction of its rays that found a surface on screen. The rest falls through to the lightmap or the probe grid.
2. Trace the previous frame's lit image, so its result can feed the current frame's lighting, and so bounce light gains a bounce each frame.
3. Denoise spatially and temporally, at half resolution, with the same edge-aware upsample as GTAO and SSR.

What Three.js has. SSGINode already exists, built on the visibility-bitmask approach of SSRT3, which extends GTAO's horizon search to gather light as well as occlusion. It outputs a GI texture and an AO texture. The SSGI example traces the current frame's beauty pass and composites over it:

const composite = vec4( scenePassColor.rgb.mul( ao ).add( scenePassDiffuse.rgb.mul( gi.rgb ) ), scenePassColor.a );

The gaps. The same as SSR's, and for the same reason.

• Coverage output. SSGINode reports how much light it found, but not how much of the hemisphere it could answer. Without coverage it can only be added on top, and on a lightmapped surface that counts the static bounce twice: once from the bake, once from SSGI reading the image the bake lit. With coverage it replaces the baked bounce where it has an answer, so the two agree on static light and SSGI contributes only what changed: light bounced off moving objects, and from moving lights.
• Previous-frame tracing. Reading the current frame's beauty pass forces SSGI to run after it, as a composite. The previous-frame tracing being added for SSR applies here unchanged, with the same rules: linear HDR before tone mapping, and clamped.
• A diffuse source. Once it has both, SSGINode becomes the first entry in the diffuse list described below, with no change to the materials.

Reflections: local probes and the cascade#

What the games require. Three.js grew up rendering a single object floating in space, lit by one environment map, and for that it is superb. Indoors, one environment map is simply wrong: walk from a red room into a blue corridor and the floor should reflect red, then blue, not the sky.

Every game in the survey, and Unreal's reflection captures, solves this with local reflection probes: cubemaps captured inside each room, each with a box describing the volume it influences. Inside its box a probe is box-projected: the reflection ray is intersected with the room's box first, and the cubemap is sampled toward that hit point instead of along the raw ray. That one step removes most of a cubemap's parallax error, so the far wall's reflection stays put on the floor as you walk (Lagarde's write-up is the classic explanation). Where boxes overlap, probes blend, and the sky beyond them is just the outermost probe: one with an infinite box and no box projection.

What Three.js has. The ingredients are there. CubeCamera captures a cubemap, PMREMGenerator prefilters it for roughness, and getParallaxCorrectNormal() box-projects it, as the box-projected environment map example shows. But that is one probe per material, set by hand. There is no notion of a set of probes in a scene, of influence volumes, or of blending between them.

The gaps, and how we are filling them.

• A probe system. Box-projected reflection probes for WebGPURenderer adds scene-level probes with influence boxes and blending, and is up as a draft PR.
• Fast access to many probes. Binding one cubemap per probe does not scale. Cube array textures put every probe in one texture; sampling them is merged, the texture class is in review.
• Probe normalization. A probe is captured at one point in a room, so it sees that room's bright, open view everywhere it is used. A dark corner or the inside of a cupboard then reflects a brightly lit room. Call of Duty's fix, introduced in Black Ops II and carried into Infinite Warfare, is to scale the probe's reflection by the ratio of the diffuse light at the pixel to the diffuse light at the probe. The lightmaps and probe grid already provide both, so it costs almost nothing. It is still to do.
• Small probes. PMREMGenerator has a minimum size of 256². A 128² probe is plenty for blurry room reflections, since SSR in front of it handles the sharp ones, and it takes a quarter of the memory. Prefiltering at 128² is still to do.

Two connected rooms, each reflecting its own walls, from three box-projected probes.

Indirect lighting in the BRDF#

What the games require. The cascades only work if the bounced light goes through the material's BRDF, so Fresnel, roughness and metalness apply to it exactly as they do to direct light. In a forward renderer, that means the beauty pass needs two inputs:

• Indirect diffuse: SSGI where it is enabled and has coverage; then the lightmap on static meshes; on everything else, the probe grid, falling back by coverage to the environment map outside the grid's volume; multiplied by AO.
• Indirect specular: SSR, falling back by coverage to the box-projected probes, falling back to the environment map, with GTSO applied to the fallback part.

Both then go through the usual split-sum approximation (Karis 2013) alongside the direct lights.

Engines differ in where this happens. Deferred engines like Unreal sample lightmaps while filling the G-buffer and composite reflections in a separate full-screen pass. Three.js renders forward, so the natural fit is to compute everything screen-space, SSR, SSGI and AO, before the beauty pass and let each material read it while it shades. Tracing the previous frame is what makes that order possible.

What Three.js has. Indirect diffuse is mostly in place: lightMap, LightProbeGrid and scene.environment all feed it, and AO already reaches it through builtinAOContext. Indirect specular comes only from scene.environment, or a material's own envMap.

The gap. This is the biggest missing piece. There is no way to plug a screen-space result, or a cascade of sources, into a material's indirect specular, the way AO is plugged into its diffuse. What we want is the AO pattern, extended to both terms. Something like this, where the names are proposals and not an API that exists yet:

scenePass.contextNode = indirectLightingContext( {
	diffuse: [
		ssgiPass.getDiffuseSource(),                 // optional: what the screen can see
		lightMapDiffuse(),                           // then static meshes
		lightProbeGridDiffuse( lightProbeGrid )      // then dynamic ones
	],
	specular: [
		ssrPass.getSpecularSource(),                 // what the screen can see
		reflectionProbesSpecular( reflectionProbes ) // then the room
	],
	occlusion: aoPass                                // AO and GTSO, skipped for screen-space hits
} );                                                 // the environment map is the last fallback in both

With the default being today's behaviour, the environment map for both, nothing changes for the single-object scenes Three.js already does well. Indoor scenes opt in to the full cascade. The full design is in Pluggable Indirect Lighting for Three.js.

TRAA closes the loop#

What the games require. Temporal anti-aliasing is standard in every modern engine. Here it does double duty: its output is next frame's input to SSR, so it has to stay sharp under motion and must not feed ghosts or fireflies back into the reflections.

What Three.js has. A good TRAA node with Halton jitter, motion-vector reprojection and variance clipping of the history.

This is in good shape, and no major gap stands in the way. Two refinements remain:

• Sharper history. History sampled with a bilinear filter blurs a little more every frame the camera moves. Catmull-Rom history sampling, a draft PR, keeps it sharp.
• Wider clipping at rest. Variance clipping is what stops ghosting: it rejects history that no longer matches the current frame. But it also rejects good history. TRAA already tightens its clip box to 0.5σ under fast motion, but at rest the box was 1σ, tight enough to clip converged sub-pixel detail, so a still image never fully resolved and longer histories actually scored worse. A drafted change, not yet submitted, widens the box to 2σ at rest and adds a historyLength control. Measured with the TRAA harness described below, a static view gains 2.15 dB of PSNR against the supersampled reference and a still camera with moving objects gains 1.3 dB, while fast camera motion is unchanged to within 0.06 dB.

Debug harnesses, not hope#

Some of the most valuable work on this project has not been rendering features at all, but debug harnesses: small examples that test one stage in isolation, against a known answer.

The reason is that screen-space techniques are nearly impossible to debug in a real scene. By the time an SSR result reaches the screen, it has been traced, denoised, reprojected, pushed through the BRDF, added to everything else and anti-aliased. If the reflections look a little blotchy, which stage is at fault? Looking at a finished frame and tweaking parameters is mostly hoping it works.

So each stage gets its own harness:

• Noise. The noise smoothing harness, added with Houston Stratified Noise, feeds a noise function through the same kind of spatial blur and temporal accumulation a real effect uses, and shows how quickly each noise function reconstructs a smooth result. That is the question that actually matters, and it is how we chose the noise for SSR.
• Denoising. The denoiser harness, which I opened today, takes a known image and samples it the way SSR does: sparse, spatially jittered samples, with an adjustable coverage level to imitate rays that miss. It feeds those samples through the spatial and temporal denoiser and compares the result with the original image, with an error view showing where it is too bright or too dark.
• Anti-aliasing. The TRAA harness, drafted and not yet submitted, compares TRAA with an unjittered 4× supersampled reference under isolated motions: static, pans, orbit, yaw, roll, dolly, zoom, sub-pixel drift and moving objects. It reports PSNR for each, with a delta view and controls for every TRAA parameter.

The denoiser harness: the reconstructed image (left) and the sparse, jittered samples it was built from (right).

With a known reference, a denoiser change is either measurably better or it is not, and it can be tuned before it ever touches a reflection.

The harnesses are just as useful for review. A reviewer does not have to take a before-and-after screenshot on trust, or judge a subtle difference by eye in a busy scene: they can open the harness and see the error against a known answer. The aim is for each screen-space stage in this pipeline to arrive with one.

Future work#

A few threads are worth pulling once the pipeline is complete.

• Cheaper noise. Houston Stratified Noise gives SSR excellent sampling without downloading a texture, but it costs about 400 operations per sample, and every pixel pays that every frame. That is too much for something so basic. The promising trade is to shift some of the work into a small precomputed map: a few hundred kilobytes, generated at startup in 50 ms or less, that keeps HSN's properties while making each sample a lookup and a few adds.
• Auto-exposure. Walking from a dark room into a bright courtyard needs eye adaptation, as every game has. Three.js does not have it yet.
• Reflection probes in glTF. KHR_lightmap carries lightmaps, but there is no standard way to ship reflection probes. The minimum is each probe's position and influence box, so a loader can capture the probes on load; a fuller version would also carry the prefiltered cubemaps, so nothing needs capturing at all.
• Directional lightmaps. The lightmaps here store a single irradiance value per texel, so normal maps cannot respond to baked light. Directional lightmaps restore that response for a few more bytes per texel: an ambient term plus one dominant direction as in The Last of Us and Call of Duty, L1 spherical harmonics as in Frostbite, or spherical Gaussians as in The Order: 1886.

Where this lands#

When the remaining pieces are in, a Three.js indoor scene will be lit the way DOOM, Call of Duty and Killzone light theirs: baked bounce light that is soft and correct, probes that carry it to moving objects, occlusion in every corner, and reflections that are sharp where the screen can see them and right for the room where it cannot. It will run in real time on ordinary hardware, from a code base small enough to ship with a web page and assets small enough to download in seconds.

Nothing here is invented. Every technique has shipped in games played by millions of people. What is new is bringing them together, in one coherent pipeline, to the web's most popular 3D library, just as more people than ever are using it.

How you can help#

Most of this pipeline is sitting in open pull requests, and review time is the scarce resource. The most useful thing you can do is try them:

• Follow along. I post each PR as I submit it, with pictures, on X, Bluesky and LinkedIn.
• Try the PRs on your own scenes. Most of the PRs linked above add or update an example. Check out the branch, point the example at an interior of your own, and see what happens.
• Tell the Three.js project what you found. Constructive feedback on the PR, whether it worked, where it broke, how it performed on your hardware, is what helps a reviewer say yes, and it makes the result better.

If indoor scenes in Three.js that stand next to a shipped game are something you want, that is how to get them sooner.