spark/docs/procedural-splats.md

Procedural Splats

Forge makes it easy to create Gaussian splat collections procedurally, and includes some splat constructors that may be useful for tasks like creating a grid or text made of splats. The example "Procedural Splats" puts some of these to use in a scene.

Adding splats to a collection

To create a PackedSplats with custom, procedurally-derived splats use the methods pushSplat or setSplat:

const splats = new PackedSplats();
const center = new THREE.Vector3(0, 0, 0);
const scales = new THREE.Vector3(0.1, 0.1, 0.1);
const quaternion = new THREE.Quaternion();
const opacity = 1.0;
const color = new THREE.Color()
splats.pushSplat(center, scales, quaternion, opacity, color);
...

The array in PackedSplats will be resized automatically to fit any splats you add. Alternatively, you can use the construct initializer callback:

const splats = new PackedSplats({
  construct: (splats) => {
    ...
    for (let i = 0; i < NUM_SPLATS; ++i) {
      // Compute splat #i
      ...
      splats.pushSplat(center, scales, quaternion, opacity, color);
    }
  },
});

Once you've created your PackedSplats, render it to the scene via a SplatMesh:

const mesh = new SplatMesh({ packedSplats: splats });
scene.add(mesh);

Alternatively, you can create the SplatMesh and its splats in the initializer, which internally passes your constructor to its contained PackedSplats:

const mesh = new SplatMesh({
  constructSplats: (splats) => {
    for (let i = 0; i < NUM_SPLATS; ++i) {
      // Compute splat #i
      ...
      splats.pushSplat(center, scales, quaternion, opacity, color);
    }
  },
});
scene.add(mesh);

Grid

import { constructGrid } from "@worldlabsai/forge";

const grid = new SplatMesh({
  constructSplats: (splats) => constructGrid({
    splats,
    extents: new THREE.Box3(
      new THREE.Vector3(-10, -10, -10),
      new THREE.Vector3(10, 10, 10),
    ),
  }),
});
scene.add(grid);

Required parameters

Parameter	Description
splats	PackedSplats object to add splats to
extents	min and max box extents of the grid

Optional parameters

Parameter	Default	Description
stepSize	1	step size along each grid axis
pointRadius	0.01	spherical radius of each Gsplat
pointShadowScale	2.0	relative size of the "shadow copy" of each Gsplat placed behind it
opacity	1.0	Gsplat opacity
color	RGB-modulated grid	Gsplat color (THREE.Color) or function to set color for position: ((THREE.Color, THREE.Vector3) => void)

XYZ axis

import { constructAxes } from "@worldlabsai/forge";

const axes = new SplatMesh({
  constructSplats: (splats) => constructAxes({ splats }),
});
axes.position.set(0, 0, -1);
scene.add(axes);

Required parameters

Parameter	Description
splats	PackedSplats object to add splats to

Optional parameters

Parameter	Default	Description
scale	0.25	scale (Gsplat scale along axis)
axisRadius	0.0075	radius of the axes (Gsplat scale orthogonal to axis)
axisShadowScale	2.0	relative size of the "shadow copy" of each Gsplat placed behind it
origins	[new THREE.Vector3()]	origins of the axes (default single axis at origin)

Gsplat sphere

import { constructSpherePoints } from "@worldlabsai/forge";

const sphere = new SplatMesh({
  constructSplats: (splats) => constructSpherePoints({
    splats,
    maxDepth: 4,
  }),
});
scene.add(sphere);

Required parameters

Parameter	Description
splats	PackedSplats object to add splats to

Optional parameters

Parameter	Default	Description
origin	new THREE.Vector3()	center of the sphere (default: origin)
radius	1.0	radius of the sphere
maxDepth	3	maximum depth of recursion for subdividing the sphere. Warning: Gsplat count grows exponentially with depth
filter	null	filter function to apply to each point, for example to select points in a certain direction or other function ((THREE.Vector3) => boolean)
pointRadius	0.02	radius of each oriented Gsplat
pointThickness	0.001	flatness of each oriented Gsplat
color	new THREE.Color(1, 1, 1)	color of each Gsplat (THREE.Color) or function to set color for point: ((THREE.Color, THREE.Vector3) => void)

Rasterizing Text

const splats = textSplats({
  text: string;
  font?: string;
  fontSize?: number;
  color?: THREE.Color;
  rgb?: THREE.Color;
  dotRadius?: number;
  textAlign?: "left" | "center" | "right" | "start" | "end";
  lineHeight?: number;
});
scene.add(splats);

Required parameters

Parameter	Description
text	text string to display

Optional parameters

Parameter	Default	Description
font	"Arial"	browser font to render text with
fontSize	32	font size in pixels/Gsplats
color	new THREE.Color(1, 1, 1)	SplatMesh.recolor tint assuming white Gsplats
rgb	new THREE.Color(1, 1, 1)	Individual Gsplat color
dotRadius	0.8	Gsplat radius (0.8 covers 1-unit spacing well)
textAlign	"start"	text alignment: "left", "center", "right", "start", "end"
lineHeight	1.0	line spacing multiplier, lines delimited by "\n"

Turning images into Gsplats

const image = imageSplats({
  url: string;
  dotRadius?: number;
  subXY?: number;
  forEachSplat?: (
    width: number,
    height: number,
    index: number,
    center: THREE.Vector3,
    scales: THREE.Vector3,
    quaternion: THREE.Quaternion,
    opacity: number,
    color: THREE.Color,
  ) => number | null;
});
scene.add(image);

Required parameters

Parameter	Description
url	URL of the image to convert to splats (example: url: "./image.png")

Optional parameters

Parameter	Default	Description
dotRadius	0.8	Radius of each Gsplat, default covers 1-unit spacing well
subXY	1	Subsampling factor for the image. Higher values reduce resolution, for example 2 will halve the width and height by averaging
forEachSplat	undefined	Optional callback function to modify each Gsplat before it's added. Return null to skip adding the Gsplat, or a number to set the opacity and add the Gsplat with parameter values in the objects center, rgba etc. were passed into the forEachSplat callback. Ending the callback in return opacity; will retain the original opacity.

Example

// Load RGBA image from image.png, subsample it 2x
// horizontally and vertically, and create Gsplats for
// the resulting pixels that have at least 10% opacity.
const image = imageSplats({
  url: "./image.png",
  subXY: 2,
  forEachSplat: (width, height, index, center, scales, quaternion, opacity, color) => {
    // Only keep Gsplats with opacity 10% or higher
    return (opacity >= 0.1) ? opacity : null;
  },
});
scene.add(image);

Particle Effects

Some of the building blocks are meant to serve as "code inspiration", showing how a particle effect animation can be achieved using a "stateless" dyno computation graph that uses only the Gsplat index input to produce pseudo-random numbers that drive various randomized particle effects.

Note that Gsplat sorting takes a little bit of time and can "lag behind" the Gsplat updates each frame, so it's important that there is a reasonably stable correspondence between each frame's Gsplat output for the same index.

In staticBox, the AABB is sliced up into a 3D grid of cells, and random X/Y/Z points within those cells are sampled each frame, retaining consistency in Gsplat position between successive frames. See examples/gsplat-sweets-garden/main.js for an example that creates a staticBox.

generators.snowBox(...options)

Similarly, snowBox produces Gsplat trajectories that move in a deterministic fashion over time, with high similarity between adjacent frames. See examples/atmospheric/main.js for an example that creates a snowBox.

A snowBox instance has a collection of properties that can be tuned to achieve different particle effects. DEFAULT_SNOW and DEFAULT_RAIN are example parameter sets that look a lot like snow and rain, and can be used as a starting point for further tweaking: const mySnow = { ...DEFAULT_SNOW, density: 500 };

const snowControls = generators.snowBox({
  box,
  minY,
  numSplats,
  density,
  anisoScale,
  minScale,
  maxScale,
  fallDirection,
  fallVelocity,
  wanderScale,
  wanderVariance,
  color1,
  color2,
  opacity,
  onFrame,
});
scene.add(snowControls.snow);

Parameter	Default Value	Description
box	new THREE.Box3(new THREE.Vector3(-1, -1, -1), new THREE.Vector3(1, 1, 1))	min and max box extents of the snowBox
minY	Number.NEGATIVE_INFINITY	minimum y-coordinate to clamp particle position, which can be used to fake hitting a ground plane and lingering there for a bit
numSplats	calculated from box and density	number of Gsplats to generate
density	100	density of Gsplats per unit volume
anisoScale	new THREE.Vector3(1, 1, 1)	The xyz anisotropic scale of the Gsplat, which can be used for example to elongate rain particles
minScale	0.001	Minimum Gsplat particle scale
maxScale	0.005	Maximum Gsplat particle scale
fallDirection	new THREE.Vector3(0, -1, 0)	The average direction of fall
fallVelocity	0.02	The average speed of the fall (multiplied with fallDirection)
wanderScale	0.01	The world scale of wandering overlay motion
wanderVariance	2	Controls how uniformly the particles wander in sync, more variance means more randomness in the motion
color1	new THREE.Color(1, 1, 1)	Color 1 of the two colors interpolated between
color2	new THREE.Color(0.5, 0.5, 1)	Color 2 of the two colors interpolated between
opacity	1	The base opacity of the Gsplats
onFrame	undefined	Optional callback function to call each frame