Adding Global Sort to Three.js Gaussian Splatting

I added native Gaussian splat rendering to Three.js in an earlier post. GaussianSplat is the right tool for one captured object. It sorts that cloud against itself and draws it. Two overlapping GaussianSplat meshes cannot share an order, so where they intersect you get a popping seam.

GaussianSplatGroup is the fix. It packs many splat BufferGeometrys into one set of storage buffers, sorts the packed set once, and draws it with a single instanced call. The lion and tomatoes in the group example overlap on purpose so you can see the difference.

This post covers why that global sort matters, how to use the API, how the implementation stays cheap when the camera or an item matrix moves, and how far you can push it on real WebGPU hardware.

Why a group#

A product page with a splat of the object and a splat of the set dressing, a room with furniture captured as separate scans, a configurator that swaps attachments: those are several geometries, each with its own transform and visibility. Alpha blending still needs one back-to-front order across all of them.

If each cloud sorts and draws itself, the GPU finishes one entire cloud before the next. Splats from cloud A that sit behind cloud B draw on top. GaussianSplatGroup puts every live splat into one list, so a tomato in front of the lion's paw blends in the right order.

The group does not wrap scene-graph children. You pass each cloud's BufferGeometry into addSplat and get back an id, the same pattern as InstancedMesh / BatchedMesh. Layout, sorting, and draw state update inside onBeforeRender. There is no extra update() to call before renderer.render().

Usage#

GaussianSplatGroup takes the same geometry contract as GaussianSplat. Load two files, add them, set their matrices:

import * as THREE from 'three/webgpu';
import { SPZLoader } from 'three/addons/loaders/SPZLoader.js';
import { GaussianSplatGroup } from 'three/addons/objects/GaussianSplatGroup.js';

const renderer = new THREE.WebGPURenderer();
await renderer.init();

const scene = new THREE.Scene();
const camera = new THREE.PerspectiveCamera( 50, window.innerWidth / window.innerHeight, 0.01, 100 );

const loader = new SPZLoader();
const [ lionGeometry, tomatoesGeometry ] = await Promise.all( [
	loader.loadAsync( 'lion.spz' ),
	loader.loadAsync( 'tomatoes.spz' )
] );

const group = new GaussianSplatGroup( {
	autoCompact: false, // grow-only buffers; call compact() to shrink
	initialSize: 2_000_000, // preallocate for 2M splats
	shDegree: 2
} );
scene.add( group );

const lionId = group.addSplat( lionGeometry );
const tomatoesId = group.addSplat( tomatoesGeometry );

group.setMatrixAt( lionId, lionMatrix );
group.setMatrixAt( tomatoesId, tomatoesMatrix );

renderer.setAnimationLoop( () => {

	renderer.render( scene, camera );

} );

Toggle a cloud with setVisibleAt( id, false ). Remove it with deleteSplat( id ). Move it with setMatrixAt. The group requires WebGPURenderer. The forceWebGL: true WebGL2 fallback works, with the sort caveats below.

How it stays cheap#

Rewriting every splat when a cloud moves, or reallocating GPU buffers when the camera turns, would make this unusable. The group leaves splat payloads in place and updates a small record of transforms instead.

Transforms live in a record buffer#

Each packed splat keeps its original local-space center, covariance, color, and spherical harmonics. The fourth component of the center stores a record index. A small record buffer holds that cloud's 3×4 matrix and the camera position in the cloud's local space.

setMatrixAt writes 16 floats for that record. It does not copy the splat arrays. The vertex shader and the sort key both read the record and transform the center on the fly. A rotating attachment is a matrix upload, not a million transformed positions.

Camera motion updates the per-record local camera position the same way, so view-dependent spherical harmonics stay correct without touching splat data.

Sort is a counting sort, and it is skipped most frames#

The packed set is ordered with the same CountingSort GaussianSplat uses: 4096 depth bins, four passes (reset, histogram, prefix sum, scatter). The payload stays put. Only an index buffer is permuted.

A new sort runs when the view direction relative to the group changes enough (a dot-product test against the last sort direction) or when the packed layout itself changed. A small orbit can reuse the previous order for several frames.

The material and SH layout are fixed at construction#

shDegree is a constructor option, default 2. Source clouds with fewer bands get padded with neutral coefficients. Clouds with more bands are truncated. That keeps the storage layout and the NodeMaterial stable while you add, hide, and remove members, so those edits do not trigger a shader recompile.

WebGL2 fallback#

WebGPURenderer can run on WebGL2 with forceWebGL: true. Compute shaders are unavailable there, so GaussianSplatGroup runs the identical counting sort in JavaScript (CountingSort.computeCPU) and uploads the order buffer. Shading still goes through TSL, which compiles to GLSL on that backend.

The algorithm is the same O(N) histogram, running on one CPU thread. A few hundred thousand splats stay interactive. A few million will hitch. Treat WebGL2 as coverage for browsers without WebGPU, and size those scenes for the CPU sort.

Grow-only buffers#

Resizing a storage buffer means allocating a new one and copying. On the WebGL2 path it also means rebuilding PBO textures. Doing that on every visibility toggle is wasted work if you are about to show the cloud again.

autoCompact defaults to true: add/remove/visibility that changes the live total resizes the shared buffers to fit. Pass autoCompact: false and the buffers only grow. Hiding the lion leaves capacity at the high-water mark. Call compact() when you want the unused memory back.

initialSize preallocates for a peak count so adding clouds does not realloc until you exceed it. Passing initialSize implies autoCompact: false unless you set autoCompact: true as well:

const group = new GaussianSplatGroup( {
	autoCompact: false,
	initialSize: 2_000_000
} );

That is the setup I use when a scene streams clouds in and out. Size once for the expected peak, keep autoCompact off, and compact on a loading-screen or level change.

How many splats fit#

The ceiling is the GPU's per-binding storage buffer size. Web3DSurvey reports maxStorageBufferBindingSize from real WebGPU devices:

The covariance buffer is the fattest per-splat array (eight floats). Centers, packed colors, and SH bands sit in their own bindings, so they do not add to that one limit. A portable target is 4–8 million live splats on WebGPU. Going past 8 million is fine on desktop adapters that advertise 1 GB, and it will fail to allocate on the remaining ~3% that stop at 128 MB.

Those numbers are GPU memory, not fill rate. Fill rate, overdraw, and the sort itself become the constraint earlier on integrated GPUs. The group example is two modest clouds; the structure is the same at millions, with initialSize set to the peak you measured.

On WebGL2, budget for the CPU sort first. The storage-buffer limit is less relevant than frame time spent in computeCPU.

Try it#

The official example is webgpu_gaussian_splat_group. For loading a single cloud, see How to Use Three.js Native Gaussian Splats. For the renderer, counting sort, and spherical harmonics design, see Adding Native Gaussian Splatting Support to Three.js.