Adding Robust Light Baking to Three.js

The recipe that shipped games share, from Part 3, starts with baking indirect diffuse light into lightmaps for everything that does not move. Three.js could display a lightmap. It could not make one. Now it can: load a model, unwrap it, bake it, and see the bounced light in seconds, in the browser, with no round trip through Blender or a game engine.

What the games require#

An offline solver that bakes bounce light into lightmaps without leaking light through walls.
Lightmap UVs for every static mesh, with padding so charts do not bleed into each other, and seams that do not show.
A compact way to ship lightmaps, which are HDR, and a compact way to hold them in GPU memory.

What Three.js had#

When I started on this in September, the runtime side had been there for years. Every standard material takes a lightMap, read from a second UV set:

material.lightMap = bakedTexture;
material.lightMap.channel = 1; // the lightmap UV set

The lightmap example shows it, but with lighting baked ahead of time in another tool and shipped as ordinary textures. The only way to bake inside Three.js was zalo's ProgressiveLightMap (example), which accumulates jittered shadow maps into a lightmap over many frames. It produces lovely soft shadows, but no bounced light, and it reuses each mesh's existing UVs, packing whole objects into the atlas instead of unwrapping them.

Measured against the requirements, almost the whole authoring side was missing.

Unwrapping#

Most models do not come with a second, non-overlapping UV set. A real-time UV unwrapper now generates a lightmap atlas for any mesh, in the browser. It is 5 kB gzipped, sixteen times smaller than xatlas compiled to WebAssembly, and 1.6 to 10.6 times faster on test models. The trade-off is a less densely packed atlas: 22–43% filled, against 37–65% for xatlas. It runs in the browser and in Node.js, so an atlas can be generated on load or in a build step.

The Three.js UV unwrapper generating a chart atlas for the Damaged Helmet model, with each chart shown in a different color

The unwrapper's lightmap atlas for the Damaged Helmet, one color per chart.

Baking#

The lightmap baker is a framework for baking indirect diffuse into those UVs, with pluggable solvers. It is progressive and runs on the GPU. It takes any function that returns irradiance at a point in the world, so every source of diffuse global illumination in Three.js can feed it; it already works with light probes and VXGI as well as the solver below.

It has two export modes. Indirect only bakes the bounce light and leaves direct light and specular highlights live, which is the split the games in Part 3 use. All diffuse bakes everything into the lightmap for use with unlit materials, which is the cheapest possible frame.

A solver that does not leak#

zalo's ray radiosity is the solver that makes the baker worth using. It traces against three-mesh-bvh, so light cannot pass through walls, and bakes a Cornell box in under a second. His denoiser turns a fast, noisy bake into a clean one.

A courtyard baked with ray radiosity at 64 samples, alternating between the raw noisy result and the denoised result

zalo's courtyard at 64 samples per texel, raw and denoised.

Seams#

Where a surface is split across two UV charts, the two sides are baked separately and can disagree, which shows as a visible line. zalo's seam stitching and gutter filling fixes this and is in review.

Storing and shipping lightmaps#

A baked lightmap is only useful if it can be saved, downloaded and held in memory cheaply. This part is not about lighting at all. It is about storage, and most of it is glTF.

So far glTF has been optimized for the product viewer: one object, its materials, and an environment map supplied by the page. It does that very well, in the same way Three.js does. But it has no place for baked lighting, because a product viewer does not need one. The next level of web games will use the pipelines and workflows of AAA game engines, and they need a storage format that can carry what those pipelines produce: lightmaps, HDR textures, and in time reflection probes. It is time to move glTF there.

Lightmaps in glTF#

There is no standard glTF extension for lightmaps. I started with the one that exists in the wild: MOZ_lightmap, the Mozilla Hubs extension, in production since 2020 but never fully specified. I wrote up its specification and a Three.js loader and exporter for it. Review at Khronos showed where it falls short: it has no atlas offset and scale, no definition of HDR, and no defined place in the PBR material model.

So we have proposed KHR_lightmap, currently a draft. It stores irradiance in physical units, contributes to diffuse light only, and binds lightmaps per node and per GPU instance, so a mesh repeated forty times can share one mesh and one material and still have its own region of the atlas. That avoids the common trick of baking into emissive, which breaks materials and counts light twice.

A Cornell box loaded from a glTF file in Three.js, with baked indirect diffuse lighting from a lightmap and a live point light supplying direct light and a specular highlight

A Cornell box loaded from glTF: the bounce light is baked, and one live light supplies direct light and the highlight.

HDR textures in glTF#

Lightmaps need HDR, which rules out PNG, and EXR or HDR files are large. UltraHDR JPEG and JPEG XL are both HDR and very small over the wire. UltraHDR works in every browser today, because it is an ordinary JPEG plus a gain map, so it is the near-term choice. JPEG XL compresses better and is the long-term one, but browser support is still limited. Three.js already loads UltraHDR, and I recently added HDR JPEG XL loading with exactly this in mind, for both lightmaps and environment maps, so it is ready when browsers are. I have drafted glTF extensions for both, Ultra HDR JPEG and linear JPEG XL; they are not yet proposed to Khronos.

A Radiance HDR environment map next to the same image loaded natively from a 32-bit float JPEG XL file in Three.js

An HDR source (left) and the same image as JPEG XL, decoded natively by the browser (right).

Shipping before ratification#

Khronos review can take years, and none of this has to wait for it. Three.js already supports EXT_texture_avif while its proposal is still open. The same path works here: publish the specifications, show working loaders and exporters, gather feedback from other implementers, and ship through loader plugins in the meantime.

GPU memory#

A small JPEG still decodes to a large texture: a 4096² lightmap is 134 MB as half-float. Encoding it to BC6H, the GPU's compressed HDR format, cuts that to 17 MB. It can be done once at load with a real-time compute encoder (Narkowicz, Real-Time BC6H Compression on GPU; there is a WGSL port), writing the blocks to a buffer and copying them into the texture. Where BC6H is not available, rg11b10ufloat halves the memory instead.

Where this lands#

Together, the unwrapper, the baker and ray radiosity are a complete in-browser baking pipeline. With seam stitching, the glTF extensions and BC6H in place, Three.js has baked global illumination that runs in the browser, ships quickly and uses little memory on the device that displays it.