Pluggable Indirect Lighting for Three.js

In The Quest for AAA Quality in Three.js I laid out a pipeline for indoor scenes: lightmaps, light probes, ambient occlusion, screen-space reflections, reflection probes and temporal anti-aliasing. Most of those pieces are in Three.js, in review or drafted.

One piece is not, and everything else depends on it. A material in Three.js gets its indirect light from fixed places. There is no way to tell it "take reflections from SSR, then from the room's probes, then from the sky". That is the part of the quest that decides whether the rest adds up to one pipeline or stays a set of separate techniques, so it gets its own post.

This is a proposal, not a decision. It describes the design I think Three.js should end up with, informed by proofs of concept I have built. What I want from Three.js maintainers and contributors is agreement on the shape before I turn it into pull requests, and I expect the shape to change.

Three terms, used throughout:

• Indirect diffuse: bounced light arriving from all directions. In shader terms, irradiance.
• Indirect specular: bounced light seen in the mirror direction, blurred by roughness. Reflections. In shader terms, radiance.
• Coverage: how much of a pixel a technique could answer, from 0 to 1.

Two slots#

A physically based material has two inputs for indirect light, one for each of those terms. I will call them slots.

Three.js fills both today, and fills them well for a single object. scene.environment, prefiltered by PMREMGenerator, feeds both slots. material.lightMap and the LightProbeGrid addon add to the diffuse slot. Ambient occlusion reaches both through builtinAOContext, and builtinGIContext lets a screen-space or voxel pass add irradiance to the diffuse slot. SSRNode and SSGINode produce excellent screen-space reflections and bounce light, but their examples add them to the finished image instead of feeding them to the material.

What an application cannot do is choose. Here is what can fill each slot.

Indirect diffuse

Indirect specular

"Objects" is whether the source can light things that move. "Lighting" and "Reflections" are whether the result updates when the scene changes.

The SDFGI row follows Godot's documentation, which is the implementation most people mean by the name. The radiance cascades row describes the version that shipped in Path of Exile 2. The paper also describes world-space variants, which would sit beside VXGI at medium scale, but calls them experimental.

Several rows in each table do the same job. VXGI and SDFGI both answer "bounce light that follows moving lights, beyond the screen", at different costs and with different limits. SSGI and radiance cascades both work from what is on screen. Lightmaps and probes both answer "static bounce light", for different kinds of object. An application has to choose, and the right choice depends on whether its lights move, how big its world is and what hardware it targets.

What games ship#

Shipped games and engines show how wide that choice is. Each row links to the talk, documentation or frame breakdown it comes from. A dash means that source does not cover that half, not that the game has nothing there.

Each of these is a sensible answer to a different question.

Unreal's baked path, Call of Duty and DOOM all bake. Unreal's documentation recommends it where lighting does not need to change, and the Call of Duty talk gives the other reason: the alternatives were too expensive for a 60 Hz game or not good enough. Even so, the three split static and dynamic objects differently. Call of Duty lightmaps only its largest surfaces and lights everything else from probes.

GTA V re-renders a small cubemap of the scene every frame. The breakdown points out what makes that affordable: the game already has a very low-polygon version of the city for distant detail. Cars and characters are left out of the cubemap, so your car reflects the street but not the car next to it.

CRYENGINE's SVOGI trades baking for a voxel structure built at run time. By default it supplies only diffuse light, and the documentation says light probes are still needed for specular. That is two different sources for the two slots, in one engine, by design.

Godot ships three diffuse systems and tells users to pick one: SDFGI as the starting point for desktop games, baked lightmaps and reflection probes as the only options on mobile.

Path of Exile 2 uses radiance cascades, a newer technique from Alexander Sannikov at Grinding Gear Games. His paper says the shipped version works in screen space, a choice motivated by the game's fixed camera.

Frostbite and Unity's HDRP describe the specular side most plainly. Frostbite's course notes give it as pseudocode: evaluate SSR, then local probes while coverage is below 1, then the distant probe. HDRP calls it a reflection hierarchy: SSR sets its own weight, each probe has a weight, the sky has a fixed weight of 1, and the renderer walks the list until the total reaches 1. Lumen and HDRP's SSGI apply the same shape to diffuse: trace the screen first, and hand whatever it misses to something slower or coarser.

No two rows are the same, and no row is wrong. That is the argument for making this configurable in Three.js. A product viewer, an architectural walkthrough and a game with moving lights need three different stacks. A library can serve all three only if the stack is something the application sets.

How the sources combine#

The quest post explains the cascade in detail, so here is the short version.

Each source returns a color and a coverage. The first source in the list contributes its color times its coverage. The next source fills whatever is left, 1 − coverage, and so on down the list. For specular the list is typically SSR, then reflection probes, then the environment map. SSR has high coverage where the reflected surface is on screen and none where it is not. A probe has coverage inside its box. The environment map always has coverage 1, so the list always ends with an answer.

Diffuse is the same idea: the lightmap on static meshes, the probe grid on dynamic ones, the environment for anything else, with a screen-space or voxel result in front if there is one.

Coverage is what lets a screen-space source sit in front of a baked one. SSGI traces the lit image, and on a lightmapped surface that image already contains the baked bounce. Added on top, as an extra irradiance term, SSGI counts that bounce twice. As the first source in the list, it replaces the lightmap by its coverage instead: the two agree wherever nothing has changed, and SSGI contributes what the bake could not know, light bounced off moving objects and from moving lights. That needs SSGINode to report coverage, which it does not do yet.

Two more rules matter.

Occlusion applies to the sources that need it. Ambient occlusion scales indirect diffuse from lightmaps, probes and the environment. Specular occlusion, ideally computed from a bent normal, scales the probe and environment part of indirect specular. A screen-space hit is not occluded again: an SSR or SSGI ray that found a surface has already accounted for what is in the way. Direct light is never touched, because it has shadows.

Everything goes through the material's BRDF. The slots carry incoming light, not finished pixels. The material applies Fresnel, roughness and metalness to it, exactly as it does for direct light. This is the difference from compositing a reflection over the finished image, where it bypasses the lighting model.

The proposed design#

None of this is merged, so every new name below is a proposal. Names that exist in Three.js today, such as builtinAOContext, LightProbeGrid and SSRNode, mean what they mean today. The design targets WebGPURenderer and TSL, the Three.js shading language.

Configuring a stack#

One call installs both slots and their occlusion:

scenePass.contextNode = indirectLightingContext( {
	diffuse: [ lightMapDiffuse(), lightProbeGridDiffuse( lightProbes ) ],
	specular: [ ssrPass.getSpecularSource(), reflectionProbesSpecular( probes ) ],
	occlusion: aoPass
} );

Each slot is an ordered list of sources. The scene's environment map is the last fallback in both lists without being written. occlusion takes what builtinAOContext takes today, or a GTAO pass that can also supply a bent normal for specular occlusion.

All three keys are optional. A slot that is left out keeps today's behaviour for that slot.

The default changes nothing#

If an application never installs this context, no new code runs. scene.environment feeds both indirect diffuse and indirect specular, material.lightMap and LightProbeGrid add to diffuse, builtinAOContext and builtinGIContext work as they do now, and every existing example renders the same pixels. Everything here is opt-in.

What a source is#

A source is a small object with a sample function and a few declared properties:

const source = {
	sample: ( query, builder ) => vec4( light, coverage ),
	includesOcclusion: false   // true for screen-space results
};

The query is in world space. For diffuse it holds position and normal. For specular it adds viewDirection, direction, roughness and lobe.

The contract:

• Diffuse RGB is irradiance, before the material's color is applied. Specular RGB is filtered incoming radiance, before Fresnel.
• RGB is not premultiplied by coverage. The list does the weighting.
• Coverage means validity, not brightness. A valid black sample has coverage 1 and blocks the sources after it. A lightmapped corner that is truly dark must stay dark.
• No data means zero coverage, and the next source takes over.
• A source answers only the lobes it can. A clearcoat layer queries with its own direction and roughness. An SSR buffer holds one reflection per pixel, for the base layer, so the SSR source returns zero coverage for any other lobe and the probes answer instead. That decision is made while the shader is built and costs nothing at run time.

Resolving a list is the cascade, plus occlusion per source:

let remaining = 1;

for ( const source of sources ) {

	const s = source.sample( query, builder );
	const weight = remaining * s.a;
	const visibility = source.includesOcclusion ? 1 : occlusion;

	result += s.rgb * weight * visibility;
	remaining -= weight;

}

Applying occlusion inside the loop, instead of to the slot's total, is what lets an SSR hit skip specular occlusion while the probe behind it does not.

Static and dynamic objects in one list#

The games above use lightmaps on static meshes and probes on moving ones. One list handles both, because coverage can depend on the material being shaded.

lightMapDiffuse() takes no arguments. It reads material.lightMap. On a material that has a lightmap it returns that lightmap with coverage 1. On a material without one it returns coverage 0, so the probe grid behind it lights the object. This is decided per material when its shader is built, so a lightmapped mesh never samples the probe grid and a dynamic mesh never samples a lightmap.

material.lightMap stays the one way to assign a lightmap, which is what loaders and existing scenes already do. The only change is where it enters the lighting: when a diffuse list is installed, a lightmap contributes through lightMapDiffuse() at its place in the list, and is not also added on its own.

Turning sources on and off#

The list is fixed when a material's shader is built, which keeps shaders free of branches for sources that are not used. Changing the list rebuilds the affected materials, the same cost as adding a light today.

Fading or toggling a source at run time does not need that. Every packaged source has a weight uniform that scales its coverage, so setting it to 0 hands the pixel to the next source with no recompile.

Where the code lives#

Core gets the mechanism and the two sources core already owns. Everything else is an addon, next to the technique it wraps.

The changes to existing core files are small and all conditional on a list being installed. NodeMaterial takes indirect diffuse from the diffuse list. EnvironmentNode takes radiance from the specular list instead of sampling the environment directly, for the base layer and for clearcoat. PhysicalLightingModel skips its own occlusion multiply for a slot whose list has already applied it. The split-sum BRDF that turns incoming light into a shaded pixel is not changed.

Three forks in the road#

On three points there are two defensible designs. Here is each pair, and which way I lean.

How the stack is installed

I lean toward the context. It needs the least new machinery, and a scene-level property can be added later as a thin wrapper over it.

What a source is

I lean toward the object, with a bare function accepted as shorthand for a source with default properties.

Whether the environment is appended automatically

I lean toward automatic.

Four configurations#

A product viewer. Nothing to write. This is the default, and it is what Three.js does today:

scene.environment = environmentTexture;

An interior with baked light. Lightmaps on the static meshes, the probe grid on everything that moves, SSR falling back to reflection probes, and the environment behind both:

scenePass.contextNode = indirectLightingContext( {
	diffuse: [
		lightMapDiffuse(),                     // meshes that have a lightmap
		lightProbeGridDiffuse( lightProbes )   // everything else inside the grid
	],
	specular: [
		ssrPass.getSpecularSource(),           // what the screen can see
		reflectionProbesSpecular( probes )     // then the room
	],
	occlusion: aoPass                          // skipped for SSR hits
} );

A scene with moving lights. Swap the diffuse list and leave the rest alone:

	diffuse: [ vxgiPass.getDiffuseSource() ],

Adding SSGI to a baked interior. Put it at the front of the diffuse list, and leave the lightmap and probe grid behind it to answer what the screen cannot see:

	diffuse: [ ssgiPass.getDiffuseSource(), lightMapDiffuse(), lightProbeGridDiffuse( lightProbes ) ],

In both cases the specular list, the materials and the rest of the frame do not change. That is the property I care most about: changing the GI technique is a one-line change, not a different renderer.

SSR, box-projected reflection probes and the environment map resolved by coverage in one Three.js corridor.

How to get there#

This can land as small pull requests, each useful and reviewable alone.

1. The mechanism, in core. indirectLightingContext, the list resolve, and the implicit environment fallback, with tests showing that the default path is untouched and that an empty list reproduces today's environment-only result.
2. lightMapDiffuse(), with the per-material coverage rule.
3. Per-source occlusion and the specular occlusion hook, building on the bent normals work already in review.
4. lightProbeGridDiffuse(), in the LightProbeGrid addon. With steps 2 and 3 this completes baked diffuse for static and dynamic objects.
5. reflectionProbesSpecular(), with the reflection probe system already in review.
6. getSpecularSource() on SSRNode, once it outputs coverage and traces the previous frame.
7. getDiffuseSource() on SSGINode and VXGINode, as an alternative to compositing or builtinGIContext in their examples. For SSGINode this waits on the same two changes as SSR: a coverage output, and tracing the previous frame so it can run before the beauty pass.
8. An interior example that uses the whole stack.

Only the first three touch src/. The rest are addons.

Open questions#

A few things are the maintainers' call.

1. Names. indirectLightingContext, "source", getDiffuseSource() and the rest are placeholders for whatever fits Three.js conventions best.
2. The future of builtinGIContext. It adds irradiance on top of the environment. A diffuse source replaces part of it by coverage. Both can coexist, but an additive screen-space term double-counts the bounce already baked into a lightmap, which a source avoids. Should the older one eventually be deprecated in favour of sources?
3. Scope across materials. The diffuse slot can serve every lit node material. The specular slot is only meaningful for the physical ones. Is that the right line?

What I am asking for#

Nothing here changes what Three.js does by default, and nothing here needs to land at once. The core of it is one small file and a few conditionals in the lighting path. The sources can then arrive one at a time, as addons, each reviewed on its own merits.

If you maintain or contribute to Three.js, I would value your view on the forks and questions above before the first pull request. If you build with Three.js, tell me which stack you would configure. The point of the design is that your answer does not have to match mine.