mirror of
https://github.com/storytold/spark.git
synced 2026-10-09 00:09:53 +00:00
Renamed "forge" to "spark". Updated meta-references and future domain. (#40)
This commit is contained in:
Andreas Sundquist
committed by
GitHub
GitHub
parent
ea9e4e3da9
commit
acb541cc4e
@@ -1,16 +1,16 @@
|
||||
# Community Resources
|
||||
|
||||
A collection of resources and examples created by the community. To add your own, please submit a PR to edit this page at [forge-gfx/forge](https://github.com/forge-gfx/forge/tree/main/docs/docs/community-resources.md).
|
||||
A collection of resources and examples created by the community. To add your own, please submit a PR to edit this page at [sparkjs-dev/spark](https://github.com/sparkjs-dev/spark/tree/main/docs/docs/community-resources.md).
|
||||
|
||||
## Discord
|
||||
|
||||
Join the [Forge Discord](https://discord.gg/W39qmSKemS) to connect with other users and developers.
|
||||
Join the [Spark Discord](https://discord.gg/W39qmSKemS) to connect with other users and developers.
|
||||
|
||||
## React Examples
|
||||
|
||||
Forge can be used alongside or within React for declarative scene management, dynamic rendering, and state management between your user interface and the 3D scene. See the following examples for how to use Forge with React.
|
||||
Spark can be used alongside or within React for declarative scene management, dynamic rendering, and state management between your user interface and the 3D scene. See the following examples for how to use Spark with React.
|
||||
|
||||
- [`forge-react-basic`](https://github.com/forge-gfx/forge-react-basic): A basic example of creating a `<canvas>` and THREE.js scene with Forge.
|
||||
- [`forge-react-r3f`](https://github.com/forge-gfx/forge-react-r3f): Use Forge declaratively in React with [React Three Fiber](https://r3f.docs.pmnd.rs).
|
||||
- [`forge-react-router`](https://github.com/forge-gfx/forge-react-router): An example of using Forge and React Three Fiber with [React Router](https://reactrouter.com) v7 framework mode with SSR.
|
||||
- [`forge-react-nextjs`](https://github.com/forge-gfx/forge-react-nextjs): An example of using Forge and React Three Fiber with Next.js App Router.
|
||||
- [`spark-react-basic`](https://github.com/sparkjs-dev/spark-react-basic): A basic example of creating a `<canvas>` and THREE.js scene with Spark.
|
||||
- [`spark-react-r3f`](https://github.com/sparkjs-dev/spark-react-r3f): Use Spark declaratively in React with [React Three Fiber](https://r3f.docs.pmnd.rs).
|
||||
- [`spark-react-router`](https://github.com/sparkjs-dev/spark-react-router): An example of using Spark and React Three Fiber with [React Router](https://reactrouter.com) v7 framework mode with SSR.
|
||||
- [`spark-react-nextjs`](https://github.com/sparkjs-dev/spark-react-nextjs): An example of using Spark and React Three Fiber with Next.js App Router.
|
||||
|
||||
@@ -1,9 +1,9 @@
|
||||
# Controls
|
||||
|
||||
A program using `Forge` can use any camera control scheme that is compatible with THREE.js and will typically manipulate a `THREE.Camera` object's transform. `Forge` also ships with simple, intuitive controls for navigating 3D space that use the keyboard + mouse, game pad, or mobile multi-touch. To add these controls, you can create a `ForgeControls` instance:
|
||||
A program using `Spark` can use any camera control scheme that is compatible with THREE.js and will typically manipulate a `THREE.Camera` object's transform. `Spark` also ships with simple, intuitive controls for navigating 3D space that use the keyboard + mouse, game pad, or mobile multi-touch. To add these controls, you can create a `SparkControls` instance:
|
||||
|
||||
```typescript
|
||||
const controls = new ForgeControls({
|
||||
const controls = new SparkControls({
|
||||
canvas: HTMLCanvasElement;
|
||||
});
|
||||
|
||||
@@ -13,7 +13,7 @@ renderer.setAnimationLoop((time) => {
|
||||
});
|
||||
```
|
||||
|
||||
`ForgeControls` instantiates two classes `FpsMovement` and `PointerControls` that it updates internally. You can also instantiate and use these two classes separately:
|
||||
`SparkControls` instantiates two classes `FpsMovement` and `PointerControls` that it updates internally. You can also instantiate and use these two classes separately:
|
||||
|
||||
## `class FpsMovement`
|
||||
|
||||
|
||||
@@ -1,12 +1,12 @@
|
||||
# Dyno shaders
|
||||
|
||||
The `dyno` shader graph system is one of the architectural pillars of Forge, allowing you to create custom computation graphs using Javascript (and optionally GLSL) that are compiled to GLSL and run on the GPU, similar to shader graph systems in modern 3D graphics engines.
|
||||
The `dyno` shader graph system is one of the architectural pillars of Spark, allowing you to create custom computation graphs using Javascript (and optionally GLSL) that are compiled to GLSL and run on the GPU, similar to shader graph systems in modern 3D graphics engines.
|
||||
|
||||
A core component of this system is the class `Dyno` and its subclasses, which can be thought of as function blocks with multiple typed inputs and outputs. Values passed between such blocks are of type `DynoVal<T>`, where `T` must be a `DynoType`, representing a GPU type in GLSL. Using TypeScript, Forge ensures type safety and static validation of the GPU computation graph.
|
||||
A core component of this system is the class `Dyno` and its subclasses, which can be thought of as function blocks with multiple typed inputs and outputs. Values passed between such blocks are of type `DynoVal<T>`, where `T` must be a `DynoType`, representing a GPU type in GLSL. Using TypeScript, Spark ensures type safety and static validation of the GPU computation graph.
|
||||
|
||||
All `dyno` code is contained within `src/dyno`, and has definitions which cover all built-in GLSL ES 3.0 types (`"int"`, `"float"`, `"vec"`, etc.) and most of the standard functions (`mul`, `cross`, `texelFetch`, etc.). Note that regular Javascript functions can't be part of a `dyno` computation graph: instead of `x + y` you should use `dyno.add(x, y)`. You can also define your own custom types and `dyno` blocks, both by compositing existing `dyno` functions or by writing GLSL code directly.
|
||||
|
||||
Forge currently uses these Dynos in two main places:
|
||||
Spark currently uses these Dynos in two main places:
|
||||
|
||||
- Dynamically generating splats from `SplatGenerator`/`SplatMesh` into the scene
|
||||
- Computing the splat distance metric for CPU readback and sorting
|
||||
@@ -17,7 +17,7 @@ Learning to build and use `dyno` programs is probably best approached by startin
|
||||
|
||||
## `type DynoType`
|
||||
|
||||
A `DynoType` can be either a string that corresponds to a built-in GLSL type, or `{ type: "MyType" }` for a user-defined type. These types are used both for identifying value types `DynoVal<T extends DynoType>` and for declaring input/output types for `Dyno` blocks, for example `{ index: "int" }` or `{ gsplat: Gsplat }` (`Gsplat` is defined as `{ type: "Gsplat" }`). Forge uses these to enforce TypeScript constraints on inputs+outputs of `Dyno` blocks to generate correct GLSL code.
|
||||
A `DynoType` can be either a string that corresponds to a built-in GLSL type, or `{ type: "MyType" }` for a user-defined type. These types are used both for identifying value types `DynoVal<T extends DynoType>` and for declaring input/output types for `Dyno` blocks, for example `{ index: "int" }` or `{ gsplat: Gsplat }` (`Gsplat` is defined as `{ type: "Gsplat" }`). Spark uses these to enforce TypeScript constraints on inputs+outputs of `Dyno` blocks to generate correct GLSL code.
|
||||
|
||||
### Build-in types
|
||||
|
||||
@@ -25,7 +25,7 @@ A built-in GLSL type can be single-valued (`"int"`, `"uint"`, `"float"`, `"bool"
|
||||
|
||||
### Custom types
|
||||
|
||||
Forge defines a handful of custom types that are useful: `Gsplat`, `TPackedSplats`, `SdfArray`, `TRgbaArray`, and `SplatSkinning`. For example, in `src/dyno/splats.ts` we define `Gsplat` as `{ type: "Gsplat" }` along with a helper function:
|
||||
Spark defines a handful of custom types that are useful: `Gsplat`, `TPackedSplats`, `SdfArray`, `TRgbaArray`, and `SplatSkinning`. For example, in `src/dyno/splats.ts` we define `Gsplat` as `{ type: "Gsplat" }` along with a helper function:
|
||||
|
||||
```typescript
|
||||
export const defineGsplat = unindent(`
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# Dyno Standard Library
|
||||
|
||||
The Forge `dyno` system provides a standard library of `Dyno` blocks that cover most of the built-in functions in GLSL ES 3.0, including data conversion, logic, math, trigonometry, linear algebra, texture lookups, transforms, managing uniform variables, hashing & RNG, and of course managing splat data.
|
||||
The Spark `dyno` system provides a standard library of `Dyno` blocks that cover most of the built-in functions in GLSL ES 3.0, including data conversion, logic, math, trigonometry, linear algebra, texture lookups, transforms, managing uniform variables, hashing & RNG, and of course managing splat data.
|
||||
|
||||
We use the convention of PascalCase for the names of the `Dyno` classes, and camelCase for the names of equivalent helper functions that are more ergonomic to use. For example, you can equivalently write:
|
||||
```typescript
|
||||
@@ -157,7 +157,7 @@ The following math functions follow standard GLSL ES 3.0 semantics, for example
|
||||
|
||||
## Uniform variables
|
||||
|
||||
Constant values and literals in `dyno` programs should not be changed often because it incurs a recompilation. To have a variable that can be changed every frame, you can declare a "uniform". The base class for uniforms provided by Forge is `DynoUniform`, which importantly contains a type, current value, and update function.
|
||||
Constant values and literals in `dyno` programs should not be changed often because it incurs a recompilation. To have a variable that can be changed every frame, you can declare a "uniform". The base class for uniforms provided by Spark is `DynoUniform`, which importantly contains a type, current value, and update function.
|
||||
|
||||
To update a uniform, simply assign a new value to the `value` property of the uniform. Alternatively, you can construct a `DynoUniform` with an `update` function that is called for each execution. This function can either update `value` directly, or return any non-`undefined` value to have it updated.
|
||||
|
||||
@@ -193,7 +193,7 @@ Use the following helper functions for more ergonomic creation of uniforms.
|
||||
|
||||
## Hashing & Random number generation
|
||||
|
||||
When a `dyno` program executes, each invocation for a given splat/index is effectively run in parallel, separate from the rest. In order to incorporate randomness into a `dyno` program, you must use the inputs available to the program, which is often just the `index` of the splat itself. Forge provides functions to hash any scalar or vector (integer or float) into 1-4 components of either a uint32 or float, using the PCG random number generator.
|
||||
When a `dyno` program executes, each invocation for a given splat/index is effectively run in parallel, separate from the rest. In order to incorporate randomness into a `dyno` program, you must use the inputs available to the program, which is often just the `index` of the splat itself. Spark provides functions to hash any scalar or vector (integer or float) into 1-4 components of either a uint32 or float, using the PCG random number generator.
|
||||
|
||||
| **Function** | **Description** |
|
||||
|----------|-------------|
|
||||
@@ -211,7 +211,7 @@ When a `dyno` program executes, each invocation for a given splat/index is effec
|
||||
|
||||
## Splat data
|
||||
|
||||
Forge makes it easier to work with splat data by defining the GLSL struct `Gsplat` which contains the following fields:
|
||||
Spark makes it easier to work with splat data by defining the GLSL struct `Gsplat` which contains the following fields:
|
||||
|
||||
| **Field** | **Type** | **Description** |
|
||||
|----------|-------------|-------------|
|
||||
|
||||
+10
-10
@@ -2,7 +2,7 @@
|
||||
|
||||
## Quick Start
|
||||
|
||||
Copy and paste code below in an `index.html` file or remix in the [Web Playground](https://glitch.com/edit/#!/forge-dev)
|
||||
Copy and paste code below in an `index.html` file or remix in the [Web Playground](https://glitch.com/edit/#!/sparkjs-dev)
|
||||
|
||||
```html
|
||||
<style> body {margin: 0;} </style>
|
||||
@@ -10,13 +10,13 @@ Copy and paste code below in an `index.html` file or remix in the [Web Playgroun
|
||||
{
|
||||
"imports": {
|
||||
"three": "https://cdnjs.cloudflare.com/ajax/libs/three.js/0.174.0/three.module.js",
|
||||
"@forge-gfx/forge": "https://forge.dev/releases/forge/0.1.0/forge.module.js"
|
||||
"@sparkjs-dev/spark": "https://sparkjs.dev/releases/spark/0.2.0/spark.module.js"
|
||||
}
|
||||
}
|
||||
</script>
|
||||
<script type="module">
|
||||
import * as THREE from "three";
|
||||
import { SplatMesh } from "@forge-gfx/forge";
|
||||
import { SplatMesh } from "@sparkjs-dev/spark";
|
||||
|
||||
const scene = new THREE.Scene();
|
||||
const camera = new THREE.PerspectiveCamera(60, window.innerWidth / window.innerHeight, 0.1, 1000);
|
||||
@@ -24,7 +24,7 @@ Copy and paste code below in an `index.html` file or remix in the [Web Playgroun
|
||||
renderer.setSize(window.innerWidth, window.innerHeight);
|
||||
document.body.appendChild(renderer.domElement)
|
||||
|
||||
const splatURL = "https://forge.dev/assets/splats/butterfly.spz";
|
||||
const splatURL = "https://sparkjs.dev/assets/splats/butterfly.spz";
|
||||
const butterfly = new SplatMesh({ url: splatURL });
|
||||
butterfly.quaternion.set(1, 0, 0, 0);
|
||||
butterfly.position.set(0, 0, -3);
|
||||
@@ -39,11 +39,11 @@ Copy and paste code below in an `index.html` file or remix in the [Web Playgroun
|
||||
## Install with NPM
|
||||
|
||||
```shell
|
||||
npm install @forge-gfx/forge
|
||||
npm install @sparkjs-dev/spark
|
||||
```
|
||||
## Develop and contribute to Forge
|
||||
## Develop and contribute to Spark
|
||||
|
||||
Build Forge (It requires [Rust](https://www.rust-lang.org/tools/install) installed in your machine)
|
||||
Build Spark (It requires [Rust](https://www.rust-lang.org/tools/install) installed in your machine)
|
||||
```
|
||||
npm install
|
||||
npm run dev
|
||||
@@ -53,10 +53,10 @@ This will run a Web server at [http://localhost:8080/](http://localhost:8080/) w
|
||||
|
||||
## Table of Contents
|
||||
|
||||
- [Forge Overview](overview.md)
|
||||
- [Spark Overview](overview.md)
|
||||
- [System Design](system-design.md)
|
||||
- [ForgeRenderer](forge-renderer.md)
|
||||
- [ForgeViewpoint](forge-viewpoint.md)
|
||||
- [SparkRenderer](spark-renderer.md)
|
||||
- [SparkViewpoint](spark-viewpoint.md)
|
||||
- [SplatMesh](splat-mesh.md)
|
||||
- [PackedSplats](packed-splats.md)
|
||||
- [Loading Gsplats](loading-splats.md)
|
||||
|
||||
@@ -1,8 +1,8 @@
|
||||
# Loading Splats
|
||||
|
||||
Forge provides loaders for most popular splat file formats, including `.ply` (original "gsplat" format, compressed SuperSplat variant, and plain x/y/z/r/g/b point clouds) and `.spz` (Niantic open source compressed format), both auto-detected from the file contents.
|
||||
Spark provides loaders for most popular splat file formats, including `.ply` (original "gsplat" format, compressed SuperSplat/gsplat variant, and plain x/y/z/r/g/b point clouds) and `.spz` (Niantic open source compressed format), both auto-detected from the file contents.
|
||||
|
||||
Forge can also load formats `.splat` (from [`antimatter15/splat`](https://github.com/antimatter15/splat)) and `.ksplat` (from [`mkkellogg/GaussianSplats3D`](https://github.com/mkkellogg/GaussianSplats3D)) if the file type can be inferred from the URL/path extension, or set explicitly using the `fileType` property when creating a `SplatMesh` or `PackedSplats`.
|
||||
Spark can also load formats `.splat` (from [`antimatter15/splat`](https://github.com/antimatter15/splat)) and `.ksplat` (from [`mkkellogg/GaussianSplats3D`](https://github.com/mkkellogg/GaussianSplats3D)) if the file type can be inferred from the URL/path extension, or set explicitly using the `fileType` property when creating a `SplatMesh` or `PackedSplats`.
|
||||
|
||||
## Loading auto-detectable formats `.ply` and `.spz`
|
||||
|
||||
|
||||
@@ -1,16 +1,16 @@
|
||||
*Forge aims to expand what's possible with Gaussian Splatting and help 3D/4D creators bring their visions to life and share it with others.*
|
||||
*Spark aims to expand what's possible with Gaussian Splatting and help 3D/4D creators bring their visions to life and share it with others.*
|
||||
|
||||
# Overview
|
||||
|
||||
3D Gaussian Splatting has emerged as a frontrunner in generative AI and 3D reconstruction. By representing 3D scenes and objects as collections of tiny oriented Gaussian-shaped blobs (aka "splats"), machine learning techniques can be used to create detailed, photorealistic 3D content that can be rendered in real-time. However, 3DGS is a relatively new technique that that can't be used in many traditional triangle-based 3D mesh rendering engines. Tools for creating, editing, and rendering 3DGS are in their infancy, mostly able to only work with a single, static 3DGS object at time.
|
||||
|
||||
The web's most popular 3D graphics library THREE.js, can't render 3DGS directly. Although libraries exist to render 3DGS on the web, they each come with different limitations and don't treat 3DGS objects as first-class citizens in the scene hierarchy. Limitations include: rendering only one 3DGS object at time, incorrect occlusion between 3DGS objects, inability to dynamically modify the objects, requiring WebGPU, or slow/laggy rendering. We believe 3DGS will play an important role in future 3D/4D content creation, and we built Forge to make it easy to incorporate 3DGS into online experiences.
|
||||
The web's most popular 3D graphics library THREE.js, can't render 3DGS directly. Although libraries exist to render 3DGS on the web, they each come with different limitations and don't treat 3DGS objects as first-class citizens in the scene hierarchy. Limitations include: rendering only one 3DGS object at time, incorrect occlusion between 3DGS objects, inability to dynamically modify the objects, requiring WebGPU, or slow/laggy rendering. We believe 3DGS will play an important role in future 3D/4D content creation, and we built Spark to make it easy to incorporate 3DGS into online experiences.
|
||||
|
||||
## Forge
|
||||
## Spark
|
||||
|
||||
Forge is a dynamic 3DGS renderer built for THREE.js and WebGL2 that runs in any web browser (desktop, mobile, and WebXR). With a handful of lines of code, anyone using THREE.js can easily add 3DGS to their scenes (even by vibe coding!). By creating one or more `SplatMesh` objects and adding them to your scene, Forge will render these alongside traditional triangle-based meshes during your regular `render(scene, camera)` call. `SplatMesh` derives from `THREE.Object3D` and can be translated and rotated like any other object, placed arbitrarily in the scene hierarchy, and animated by adjusting the values each frame. A `SplatMesh` can be created from most splat file formats via the `url` parameter or by directly creating the splats one by one.
|
||||
Spark is a dynamic 3DGS renderer built for THREE.js and WebGL2 that runs in any web browser (desktop, mobile, and WebXR). With a handful of lines of code, anyone using THREE.js can easily add 3DGS to their scenes (even by vibe coding!). By creating one or more `SplatMesh` objects and adding them to your scene, Spark will render these alongside traditional triangle-based meshes during your regular `render(scene, camera)` call. `SplatMesh` derives from `THREE.Object3D` and can be translated and rotated like any other object, placed arbitrarily in the scene hierarchy, and animated by adjusting the values each frame. A `SplatMesh` can be created from most splat file formats via the `url` parameter or by directly creating the splats one by one.
|
||||
|
||||
3DGS is still in its infancy, and we expect new techniques will be developed for recoloring/relighting, animation, transitions, and other creative or interactive effects. We designed Forge to be a programmable splat engine from the ground up, giving you unprecedented control over how individual splats are generated, animated, and rendered into the scene. Similar to shader graph systems in modern 3D graphics engines, Forge allows you to compose blocks of functions (called `Dyno`s) into computation graphs that can generate splats procedurally, modify them arbitrarily, or anything other computation you can imagine, and will be converted to GLSL to run on the GPU.
|
||||
3DGS is still in its infancy, and we expect new techniques will be developed for recoloring/relighting, animation, transitions, and other creative or interactive effects. We designed Spark to be a programmable splat engine from the ground up, giving you unprecedented control over how individual splats are generated, animated, and rendered into the scene. Similar to shader graph systems in modern 3D graphics engines, Spark allows you to compose blocks of functions (called `Dyno`s) into computation graphs that can generate splats procedurally, modify them arbitrarily, or anything other computation you can imagine, and will be converted to GLSL to run on the GPU.
|
||||
|
||||
## Features
|
||||
- Integrates with THREE.js rendering pipeline to fuse splat and mesh-based objects
|
||||
|
||||
@@ -49,7 +49,7 @@ Utility functions are provided in Javascript to pack/unpack these encodings:
|
||||
packedSplats.setSplat(index, center, scales, quaternion, opacity, color);
|
||||
|
||||
// Set underlying Uint32 array directly
|
||||
import { utils } from "@forge-gfx/forge";
|
||||
import { utils } from "@sparkjs-dev/spark";
|
||||
utils.setPackedSplat(packedSplats.packedArray, index, x, y, z, scaleX, scaleY, ...);
|
||||
|
||||
// Set rotation components of underlying Uint32 array directly
|
||||
@@ -111,7 +111,7 @@ Opacity is encoded on a linear scale where 0..255 maps to 0..1.
|
||||
|
||||
### Splat center encoding
|
||||
|
||||
The center x/y/z components are encoded as float16, which provides 10 bits of mantissa, or approximately 1K steps (0.1%) of resolution between each successive power of 0 from the origin, with a range of up to 32K in distance. If most of the splats are positioned relative to the origin this provides enough positional resolution. Splats that are transformed far from the origin, however (for example when bringing multiple `SplatMesh`es together in a scene that are far apart) may lose precision when mapped to the space of `ForgeRenderer`. For scenes where the user camera may move far from the origin, you may want to tie the `ForgeRenderer` origin to your camera by adding it as a child of the camera.
|
||||
The center x/y/z components are encoded as float16, which provides 10 bits of mantissa, or approximately 1K steps (0.1%) of resolution between each successive power of 0 from the origin, with a range of up to 32K in distance. If most of the splats are positioned relative to the origin this provides enough positional resolution. Splats that are transformed far from the origin, however (for example when bringing multiple `SplatMesh`es together in a scene that are far apart) may lose precision when mapped to the space of `SparkRenderer`. For scenes where the user camera may move far from the origin, you may want to tie the `SparkRenderer` origin to your camera by adding it as a child of the camera.
|
||||
|
||||
### Splat scales encoding
|
||||
|
||||
@@ -171,7 +171,7 @@ Can be used where you need an uninitialized `THREE.DataArrayTexture` like a unif
|
||||
|
||||
To generate a large number of splats we can use the `dyno` shader graph system, which allows you to create a computation graph mapping `{ index: DynoVal<"int"> }` to `{ gsplat: DynoVal<Gsplat> }` via Javascript code, then have that synthesize GLSL code, which is finally compiled and executed in parallel on the GPU.
|
||||
|
||||
This building block is used by `ForgeRenderer` to traverse each visible `SplatMesh`/`SplatGenerator` and have it "generate" its splats into the global `PackedSplats` array managed by a `SplatAccumulator`. At its core a `PackedSplats` has the ability to run `dyno` computation graphs to produce its contents using the following methods, which are typically managed by `ForgeRenderer`:
|
||||
This building block is used by `SparkRenderer` to traverse each visible `SplatMesh`/`SplatGenerator` and have it "generate" its splats into the global `PackedSplats` array managed by a `SplatAccumulator`. At its core a `PackedSplats` has the ability to run `dyno` computation graphs to produce its contents using the following methods, which are typically managed by `SparkRenderer`:
|
||||
|
||||
### `generateMapping(splatCounts: number[]): { maxSplats, mapping[] }`
|
||||
|
||||
@@ -183,7 +183,7 @@ Ensures our `PackedSplats.target` render target has enough space to generate `ma
|
||||
|
||||
### `generate({ generator, base, count, ... })`
|
||||
|
||||
Executes a `dyno` program specified by `generator` which is any `DynoBlock` that maps `{ index: "int" }` to `{ gsplat: Gsplat }`. This is invoked from `ForgeRenderer.updateInternal()` to re-generate splats in the scene for `SplatGenerator` instances whose version is newer than last generated version.
|
||||
Executes a `dyno` program specified by `generator` which is any `DynoBlock` that maps `{ index: "int" }` to `{ gsplat: Gsplat }`. This is invoked from `SparkRenderer.updateInternal()` to re-generate splats in the scene for `SplatGenerator` instances whose version is newer than last generated version.
|
||||
|
||||
## Using dynamic PackedSplats inputs in `dyno`
|
||||
|
||||
|
||||
@@ -13,7 +13,7 @@ Each splat incurs overhead in transforming it via SplatAccumulator for sorting a
|
||||
|
||||
## `maxStdDev`
|
||||
|
||||
Adjust `ForgeRenderer.maxStdDev` (either directly on `ForgeRenderer` or via constructor options) to a value less than the default `Math.sqrt(8)`. This limits the extent of the Gaussian fall-off, which by default is approx 2.8. For VR a good value is `Math.sqrt(5)`, which is perceptually very similar to the default.
|
||||
Adjust `SparkRenderer.maxStdDev` (either directly on `SparkRenderer` or via constructor options) to a value less than the default `Math.sqrt(8)`. This limits the extent of the Gaussian fall-off, which by default is approx 2.8. For VR a good value is `Math.sqrt(5)`, which is perceptually very similar to the default.
|
||||
|
||||
## `THREE.WebGLRenderer.antialias`
|
||||
|
||||
@@ -21,4 +21,4 @@ When constructing a `THREE.WebGLRenderer` you should set `antialias: false` (def
|
||||
|
||||
## `renderer.setPixelRatio(window.devicePixelRatio)`
|
||||
|
||||
Although `ForgeRenderer` is designed to work with any `devicePixelRatio`, it may impact performance due to the increased number of pixels to render and blend. If your scene consists of mostly splats, consider whether you have enough splats to justify the high DPI rendering.
|
||||
Although `SparkRenderer` is designed to work with any `devicePixelRatio`, it may impact performance due to the increased number of pixels to render and blend. If your scene consists of mostly splats, consider whether you have enough splats to justify the high DPI rendering.
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# Procedural Splats
|
||||
|
||||
Forge makes it easy to create Gaussian splat collections procedurally, and includes some splat constructors that are useful for common tasks like creating a grid or text made of splats. The example "Procedural Splats" puts some of these to use in a scene.
|
||||
Spark makes it easy to create Gaussian splat collections procedurally, and includes some splat constructors that are useful for common tasks like creating a grid or text made of splats. The example "Procedural Splats" puts some of these to use in a scene.
|
||||
|
||||
## Adding splats to a collection
|
||||
|
||||
@@ -55,7 +55,7 @@ scene.add(mesh);
|
||||
## Grid
|
||||
|
||||
```javascript
|
||||
import { constructGrid } from "@forge-gfx/forge";
|
||||
import { constructGrid } from "@sparkjs-dev/spark";
|
||||
|
||||
const grid = new SplatMesh({
|
||||
constructSplats: (splats) => constructGrid({
|
||||
@@ -90,7 +90,7 @@ scene.add(grid);
|
||||
## XYZ axis
|
||||
|
||||
```javascript
|
||||
import { constructAxes } from "@forge-gfx/forge";
|
||||
import { constructAxes } from "@sparkjs-dev/spark";
|
||||
|
||||
const axes = new SplatMesh({
|
||||
constructSplats: (splats) => constructAxes({ splats }),
|
||||
@@ -118,7 +118,7 @@ scene.add(axes);
|
||||
## Splat sphere
|
||||
|
||||
```javascript
|
||||
import { constructSpherePoints } from "@forge-gfx/forge";
|
||||
import { constructSpherePoints } from "@sparkjs-dev/spark";
|
||||
|
||||
const sphere = new SplatMesh({
|
||||
constructSplats: (splats) => constructSpherePoints({
|
||||
|
||||
@@ -1,31 +1,31 @@
|
||||
# ForgeRenderer
|
||||
# SparkRenderer
|
||||
|
||||
## Optionally adding to your `THREE.Scene`
|
||||
|
||||
Forge internally uses a `ForgeRenderer` object in your `THREE.Scene` to perform splat rendering. Forge will automatically create a `ForgeRenderer` and add it to your scene if you don't create one yourself. For more advanced use cases such as multiple viewpoints or rendering environment maps, you can create your own `ForgeRenderer` and add it anywhere in the scene, for example at the root:
|
||||
Spark internally uses a `SparkRenderer` object in your `THREE.Scene` to perform splat rendering. Spark will automatically create a `SparkRenderer` and add it to your scene if you don't create one yourself. For more advanced use cases such as multiple viewpoints or rendering environment maps, you can create your own `SparkRenderer` and add it anywhere in the scene, for example at the root:
|
||||
```typescript
|
||||
const forge = new ForgeRenderer({
|
||||
const spark = new SparkRenderer({
|
||||
renderer: myThreeJsWebGlRenderer,
|
||||
});
|
||||
const scene = new THREE.Scene();
|
||||
scene.add(forge);
|
||||
scene.add(spark);
|
||||
```
|
||||
|
||||
## Larger scenes
|
||||
|
||||
All scene splats are accumulated by SplatAccumulator into a single global PackedSplats, whose coordinates are relative to the ForgeRenderer's origin. Splats that are far away from this origin may exhibit float16 quantization artifacts, so if you plan on moving the camera large distances you can instead add `ForgeRenderer` as a child of your `THREE.Camera`, ensuring that coordinates will have higher precision near the camera viewpoint:
|
||||
All scene splats are accumulated by SplatAccumulator into a single global PackedSplats, whose coordinates are relative to the SparkRenderer's origin. Splats that are far away from this origin may exhibit float16 quantization artifacts, so if you plan on moving the camera large distances you can instead add `SparkRenderer` as a child of your `THREE.Camera`, ensuring that coordinates will have higher precision near the camera viewpoint:
|
||||
```javascript
|
||||
const aspect = canvas.width / canvas.height;
|
||||
const camera = new THREE.PerspectiveCamera(75, aspect, 0.1, 1000);
|
||||
scene.add(camera);
|
||||
// Add ForgeRenderer as a child of camera to follow it
|
||||
camera.add(forge);
|
||||
// Add SparkRenderer as a child of camera to follow it
|
||||
camera.add(spark);
|
||||
```
|
||||
|
||||
## Creating a `ForgeRenderer`
|
||||
## Creating a `SparkRenderer`
|
||||
|
||||
```typescript
|
||||
const forge = new ForgeRenderer({
|
||||
const spark = new SparkRenderer({
|
||||
renderer: THREE.WebGLRenderer;
|
||||
clock?: THREE.Clock;
|
||||
autoUpdate?: boolean;
|
||||
@@ -37,37 +37,37 @@ const forge = new ForgeRenderer({
|
||||
blurAmount?: number;
|
||||
falloff?: number;
|
||||
clipXY?: number;
|
||||
view?: ForgeViewpointOptions;
|
||||
view?: SparkViewpointOptions;
|
||||
});
|
||||
```
|
||||
### Required parameters
|
||||
| **Parameter** | Description |
|
||||
| ------------- | ----------- |
|
||||
| **renderer** | Pass in your `THREE.WebGLRenderer` instance so Forge can perform work outside the usual render loop. Should be created with `antialias: false` (default setting) as WebGL anti-aliasing doesn't improve Gaussian Splatting rendering and significantly reduces performance.
|
||||
| **renderer** | Pass in your `THREE.WebGLRenderer` instance so Spark can perform work outside the usual render loop. Should be created with `antialias: false` (default setting) as WebGL anti-aliasing doesn't improve Gaussian Splatting rendering and significantly reduces performance.
|
||||
|
||||
### Optional parameters
|
||||
|
||||
| **Parameter** | Description |
|
||||
| ----------------- | ----------- |
|
||||
| **clock** | Pass in a `THREE.Clock` to synchronize time-based effects across different systems. Alternatively, you can set the `ForgeRenderer` properties `time` and `deltaTime` directly. (default: `new THREE.Clock`)
|
||||
| **clock** | Pass in a `THREE.Clock` to synchronize time-based effects across different systems. Alternatively, you can set the `SparkRenderer` properties `time` and `deltaTime` directly. (default: `new THREE.Clock`)
|
||||
| **autoUpdate** | Controls whether to check and automatically update splat collection after each frame render. (default: `true`)
|
||||
| **preUpdate** | Controls whether to update the splats before or after rendering. For WebXR this *must* be false in order to complete rendering as soon as possible. (default: `false`)
|
||||
| **originDistance** | Distance threshold for `ForgeRenderer` movement triggering a splat update at the new origin. (default: `1.0`) This can be useful when your `ForgeRenderer` is a child of your camera and you want to retain high precision coordinates near the camera.
|
||||
| **originDistance** | Distance threshold for `SparkRenderer` movement triggering a splat update at the new origin. (default: `1.0`) This can be useful when your `SparkRenderer` is a child of your camera and you want to retain high precision coordinates near the camera.
|
||||
| **maxStdDev** | Maximum standard deviations from the center to render Gaussians. Values `Math.sqrt(5)`..`Math.sqrt(9)` produce good results and can be tweaked for performance. (default: `Math.sqrt(8)`)
|
||||
| **enable2DGS** | Enable 2D Gaussian splatting rendering ability. When this mode is enabled, any `scale` x/y/z component that is exactly `0` (minimum quantized value) results in the other two non-zero axes being interpreted as an oriented 2D Gaussian Splat instead of the usual approximate projected 3DGS Z-slice. When reading PLY files, scale values less than e^-20 will be interpreted as `0`. (default: `true`)
|
||||
| **preBlurAmount** | Scalar value to add to 2D splat covariance diagonal, effectively blurring + enlarging splats. In scenes trained without the splat anti-aliasing tweak this value was typically 0.3, but with anti-aliasing it is 0.0 (default: `0.0`)
|
||||
| **blurAmount** | Scalar value to add to 2D splat covariance diagonal, with opacity adjustment to correctly account for "blurring" when anti-aliasing. Typically 0.3 (equivalent to approx 0.5 pixel radius) in scenes trained with anti-aliasing.
|
||||
| **falloff** | Modulate Gaussian kernel falloff. 0 means "no falloff, flat shading", while 1 is the normal Gaussian kernel. (default: `1.0`)
|
||||
| **clipXY** | X/Y clipping boundary factor for splat centers against view frustum. 1.0 clips any centers that are exactly out of bounds (but the splat's entire projection may still be in bounds), while 1.4 clips centers that are 40% beyond the bounds. (default: `1.4`)
|
||||
| **view** | Configures the `ForgeViewpointOptions` for the default `ForgeViewpoint` associated with this `ForgeRenderer`. Notable option: `sortRadial` (sort by radial distance or Z-depth)
|
||||
| **view** | Configures the `SparkViewpointOptions` for the default `SparkViewpoint` associated with this `SparkRenderer`. Notable option: `sortRadial` (sort by radial distance or Z-depth)
|
||||
|
||||
## `newViewpoint(options: ForgeViewpointOptions)`
|
||||
## `newViewpoint(options: SparkViewpointOptions)`
|
||||
|
||||
Create a new `ForgeViewpoint` for this `ForgeRenderer`. Note that every `ForgeRenderer` has an initial `forge.defaultView: ForgeViewpoint` created during construction, which is used for default canvas rendering. Calling this method allows you to create additional viewpoints, which can be updated automatically each frame (performing splat sorting every time there is an update), or updated on-demand for controlled rendering for video frame rendering or similar applications.
|
||||
Create a new `SparkViewpoint` for this `SparkRenderer`. Note that every `SparkRenderer` has an initial `spark.defaultView: SparkViewpoint` created during construction, which is used for default canvas rendering. Calling this method allows you to create additional viewpoints, which can be updated automatically each frame (performing splat sorting every time there is an update), or updated on-demand for controlled rendering for video frame rendering or similar applications.
|
||||
|
||||
## `update({ scene })`
|
||||
|
||||
If `forge.autoUpdate` is `false` then you must manually call `forge.update({ scene })` to have the scene splats re-generated.
|
||||
If `spark.autoUpdate` is `false` then you must manually call `spark.update({ scene })` to have the scene splats re-generated.
|
||||
|
||||
## `renderEnvMap({ renderer, scene, worldCenter, ... })`
|
||||
|
||||
@@ -1,13 +1,13 @@
|
||||
# ForgeViewpoint
|
||||
# SparkViewpoint
|
||||
|
||||
A `ForgeViewpoint` is created from and tied to a `ForgeRenderer`, and represents an independent viewpoint of all the scene splats and their sort order. Making these viewpoints explicit allows us to have multiple, simultaneous viewpoint renders, for example for camera preview panes or overhead map views.
|
||||
A `SparkViewpoint` is created from and tied to a `SparkRenderer`, and represents an independent viewpoint of all the scene splats and their sort order. Making these viewpoints explicit allows us to have multiple, simultaneous viewpoint renders, for example for camera preview panes or overhead map views.
|
||||
|
||||
When creating a `ForgeRenderer` it automatically creates a default viewpoint `.defaultView` that is used in the normal render loop when drawing to the canvas, and is automatically updated whenever the camera moves. Additional viewpoints can be created and configured separately:
|
||||
When creating a `SparkRenderer` it automatically creates a default viewpoint `.defaultView` that is used in the normal render loop when drawing to the canvas, and is automatically updated whenever the camera moves. Additional viewpoints can be created and configured separately:
|
||||
|
||||
## Creating a `ForgeViewpoint`
|
||||
## Creating a `SparkViewpoint`
|
||||
|
||||
```typescript
|
||||
const viewpoint = forge.newViewpoint({
|
||||
const viewpoint = spark.newViewpoint({
|
||||
autoUpdate?: boolean;
|
||||
camera?: THREE.Camera;
|
||||
viewToWorld?: THREE.Matrix4;
|
||||
@@ -30,7 +30,7 @@ const viewpoint = forge.newViewpoint({
|
||||
|
||||
| **Parameter** | Description |
|
||||
| ----------------- | ----------- |
|
||||
| **autoUpdate** | Controls whether to auto-update its sort order whenever the ForgeRenderer updates the splats. If you expect to render/display from this viewpoint most frames, set this to `true`. (default: `false`)
|
||||
| **autoUpdate** | Controls whether to auto-update its sort order whenever the SparkRenderer updates the splats. If you expect to render/display from this viewpoint most frames, set this to `true`. (default: `false`)
|
||||
| **camera** | Set a `THREE.Camera` for this viewpoint to follow. (default: `undefined`)
|
||||
| **viewToWorld** | Set an explicit view-to-world transformation matrix for this viewpoint (equivalent to `camera.matrixWorld`), overrides any `camera` setting. (default: `undefined`)
|
||||
| **target** | Configure viewpoint with an off-screen render target. (default: `undefined`)
|
||||
@@ -43,19 +43,19 @@ const viewpoint = forge.newViewpoint({
|
||||
| **sortDistance** | Distance threshold for re-sorting splats. If the viewpoint moves more than this distance, splats will be re-sorted. (default: `0.01` units)
|
||||
| **sortCoorient** | View direction dot product threshold for re-sorting splats. For `sortRadial: true` it defaults to 0.99 while `sortRadial: false` uses 0.999 because it is more sensitive to view direction. (default: `0.99` if `sortRadial` else `0.999`)
|
||||
| **depthBias** | Constant added to Z-depth to bias values into the positive range for `sortRadial: false`, but also used for culling splats "well behind" the viewpoint origin (default: `1.0`)
|
||||
| **sort360** | Set this to true if rendering a 360 to disable "behind the viewpoint" culling during sorting. This is set automatically when rendering 360 envMaps using the `ForgeRenderer.renderEnvMap()` utility function. (default: `false`)
|
||||
| **sort360** | Set this to true if rendering a 360 to disable "behind the viewpoint" culling during sorting. This is set automatically when rendering 360 envMaps using the `SparkRenderer.renderEnvMap()` utility function. (default: `false`)
|
||||
|
||||
## `dispose()`
|
||||
|
||||
Call this when you are done with the `ForgeViewpoint` and want to free up its resources (GPU targets, pixel buffers, etc.)
|
||||
Call this when you are done with the `SparkViewpoint` and want to free up its resources (GPU targets, pixel buffers, etc.)
|
||||
|
||||
## `setAutoUpdate(autoUpdate: boolean)`
|
||||
|
||||
Use this function to change whether this viewpoint will auto-update its sort order whenever the attached `ForgeRenderer` updates the splats. Turn this on or off depending on whether you expect to do renders from this viewpoint for most frames.
|
||||
Use this function to change whether this viewpoint will auto-update its sort order whenever the attached `SparkRenderer` updates the splats. Turn this on or off depending on whether you expect to do renders from this viewpoint for most frames.
|
||||
|
||||
## `async prepareRenderPixels({ scene, camera?, viewToWorld?, update?, forceOrigin? })`
|
||||
|
||||
Render out a viewpoint as a Uint8Array of RGBA values for the provided scene and any `camera`/`viewToWorld` viewpoint overrides. By default `update` is `true`, which triggers its `ForgeRenderer` to check and potentially update the splats. Setting `update` to `false` disables this and sorts the splats as they are. Setting `forceOrigin` (default: `false`) to `true` forces the view update to recalculate the splats with this view origin, potentially altering any view-dependent effects. If you expect view-dependent effects to play a role in the rendering quality, enable this.
|
||||
Render out a viewpoint as a Uint8Array of RGBA values for the provided scene and any `camera`/`viewToWorld` viewpoint overrides. By default `update` is `true`, which triggers its `SparkRenderer` to check and potentially update the splats. Setting `update` to `false` disables this and sorts the splats as they are. Setting `forceOrigin` (default: `false`) to `true` forces the view update to recalculate the splats with this view origin, potentially altering any view-dependent effects. If you expect view-dependent effects to play a role in the rendering quality, enable this.
|
||||
|
||||
Underneath, `prepareRenderPixels()` simply calls `await this.prepare(...)`, `this.renderTarget(...)`, and finally returns the result of `this.readTarget()`, a Promise to a Uint8Array with RGBA values for all the pixels (potentially downsampled if the `superXY` parameter was used). These steps can also be called manually, for example if you need to alter the scene before and after `this.renderTarget(...)` to hide UI elements from being rendered.
|
||||
|
||||
@@ -73,8 +73,8 @@ Read back the previously rendered target image as a Uint8Array of packed RGBA va
|
||||
|
||||
## `autoPoll()`
|
||||
|
||||
This is called automatically by `ForgeRenderer`, there is no need to call it! The method cannot be private because then ForgeRenderer would not be able to call it.
|
||||
This is called automatically by `SparkRenderer`, there is no need to call it! The method cannot be private because then SparkRenderer would not be able to call it.
|
||||
|
||||
## `ForgeViewpoint.EMPTY_TEXTURE`
|
||||
## `SparkViewpoint.EMPTY_TEXTURE`
|
||||
|
||||
If you need an empty `THREE.Texture` to use to initialize a uniform that is updated via `onTextureUpdated(texture)`, this static texture can be handy.
|
||||
@@ -1,6 +1,6 @@
|
||||
# Splat Editing
|
||||
|
||||
Forge provides the ability to apply "edits" to splats as part of the standard `SplatMesh` pipeline. These edits take the form of a sequence of operations, applied one at a time to the set of splats in its `packedSplats`. Each operation evaluates a 7-dimensional field (RGBA and XYZ displacement) at each splat's center point in space that derives from 1 or more Signed Distance Field shapes (such as spheres, boxes, planes, etc.), blended together and across inside-outside boundaries.
|
||||
Spark provides the ability to apply "edits" to splats as part of the standard `SplatMesh` pipeline. These edits take the form of a sequence of operations, applied one at a time to the set of splats in its `packedSplats`. Each operation evaluates a 7-dimensional field (RGBA and XYZ displacement) at each splat's center point in space that derives from 1 or more Signed Distance Field shapes (such as spheres, boxes, planes, etc.), blended together and across inside-outside boundaries.
|
||||
|
||||
The result is a an RGBA,XYZ value for each splat, which combined with SplatEditRgbaBlendMode.MULTIPLY/SET_RGB/ADD_RGBA can be used to create special effects. For example, simulating simple lighting can be done with MULTIPLY or ADD_RGBA to light up or adjust the color in regions of space, using spheres for point light sources or infinite cone for a spotlight. Using MULTIPLY with opacity=0 can be used to delete splats from a region of space. The splats can also be displaced in space using the XYZ values, and adjusted each frame to create smooth deformations in space.
|
||||
|
||||
|
||||
@@ -42,7 +42,7 @@ scene.add(splats);
|
||||
|
||||
### Optional parameters
|
||||
|
||||
You can create a `new SplatMesh()` with no options, which will create a new default instance with `.numSplats=0`. Alternatively, you can provide an input `url` to fetch and decode, `fileBytes`, or `packedSplats` (an existing collection of "packed" splats). Forge supports most splat file types, including .ply (including SuperSplat compressed), .splat, .ksplat, .spz. To load filetypes .splat and .ksplat (which can't be reliably auto-detected), use the optional `fileType` argument.
|
||||
You can create a `new SplatMesh()` with no options, which will create a new default instance with `.numSplats=0`. Alternatively, you can provide an input `url` to fetch and decode, `fileBytes`, or `packedSplats` (an existing collection of "packed" splats). Spark supports most splat file types, including .ply (including SuperSplat/gsplat compressed), .splat, .ksplat, .spz. To load filetypes .splat and .ksplat (which can't be reliably auto-detected), use the optional `fileType` argument.
|
||||
|
||||
Constructor argument callbacks can be used like `constructSplats` to create a collection of splats procedurally at initialization, `onLoad` when loading and initialization completes, `onFrame` to update state every frame. Splat effects can be injected into the standard splat processing pipeline that operate in object-space and world-space via `objectModifier` and `worldModifier` respectively.
|
||||
|
||||
@@ -83,7 +83,7 @@ Additional properties on a `SplatMesh` instance:
|
||||
| **skinning** | Optional `SplatSkinning` instance for animating splats with dual-quaternion skeletal animation. (default: `null`)
|
||||
| **edits** | Optional list of `SplatEdit`s to apply to the mesh. If `null`, any `SplatEdit` children in the scene graph will be added automatically. (default: `null`)
|
||||
| **splatRgba** | Optional `RgbaArray` to overwrite splat RGBA values with custom values. Useful for "baking" RGB and opacity edits into the `SplatMesh`. (default: `null`)
|
||||
| **maxSh** | Maximum Spherical Harmonics level to use. Forge supports up to SH3. Call `updateGenerator()` after changing. (default: `3`)
|
||||
| **maxSh** | Maximum Spherical Harmonics level to use. Spark supports up to SH3. Call `updateGenerator()` after changing. (default: `3`)
|
||||
|
||||
## `dispose()`
|
||||
|
||||
@@ -103,7 +103,7 @@ Call this whenever something changes in the splat processing pipeline, for examp
|
||||
|
||||
## `update(...)`
|
||||
|
||||
This is called automatically by `ForgeRenderer` and you should not have to call it. It updates parameters for the generated pipeline and calls `updateGenerator()` if the pipeline needs to change.
|
||||
This is called automatically by `SparkRenderer` and you should not have to call it. It updates parameters for the generated pipeline and calls `updateGenerator()` if the pipeline needs to change.
|
||||
|
||||
## `raycast(raycaster, intersects: { distance, point, object}[])`
|
||||
|
||||
|
||||
@@ -4,27 +4,27 @@ One of the biggest challenges in real-time Gaussian splat rendering is sorting t
|
||||
|
||||
## Rendering data flow cycle
|
||||
|
||||
`ForgeRenderer` is a key component in Forge that manages this process. It traverses the visible THREE.js scene graph and compiles a complete list of all splats across the scene, generated by instances of `SplatMesh` in the scene hierarchy.
|
||||
`SparkRenderer` is a key component in Spark that manages this process. It traverses the visible THREE.js scene graph and compiles a complete list of all splats across the scene, generated by instances of `SplatMesh` in the scene hierarchy.
|
||||
|
||||
Each `ForgeRenderer` has a default `ForgeViewpoint` that reads back a list of all splat viewpoint distances from the GPU, then determines the splat draw order using an efficient bucket sort algorithm, run in a background worker thread via `SplatWorker`. You can spawn additional `ForgeViewpoint`s to create multiple simultaneous render viewpoints.
|
||||
Each `SparkRenderer` has a default `SparkViewpoint` that reads back a list of all splat viewpoint distances from the GPU, then determines the splat draw order using an efficient bucket sort algorithm, run in a background worker thread via `SplatWorker`. You can spawn additional `SparkViewpoint`s to create multiple simultaneous render viewpoints.
|
||||
|
||||
Finally, on the next THREE.js render() call, `ForgeRenderer` invokes a single instanced geometry draw call to draw all the scene's splats in the correct back-to-front order, merging with other opaque THREE.js geometry using the Z buffer. The sort order lags the render by at least one frame, but possibly more on older devices, but is not usually perceptible.
|
||||
Finally, on the next THREE.js render() call, `SparkRenderer` invokes a single instanced geometry draw call to draw all the scene's splats in the correct back-to-front order, merging with other opaque THREE.js geometry using the Z buffer. The sort order lags the render by at least one frame, but possibly more on older devices, but is not usually perceptible.
|
||||
|
||||
```typescript
|
||||
// Optionally add a ForgeRenderer to the scene to manage SplatMesh rendering.
|
||||
// If none is created, Forge will create one for you automatically.
|
||||
const forge = new ForgeRenderer({ renderer: webGlRenderer });
|
||||
scene.add(forge);
|
||||
// Optionally add a SparkRenderer to the scene to manage SplatMesh rendering.
|
||||
// If none is created, Spark will create one for you automatically.
|
||||
const spark = new SparkRenderer({ renderer: webGlRenderer });
|
||||
scene.add(spark);
|
||||
```
|
||||
|
||||
This design allows splats from distinct objects/scenes to coexist in space and sort correctly w.r.t. each other's splats. Splats from independent `SplatMesh`es are aggregated using a `SplatAccumulator`, which produces a `PackedSplats`, a collection splats stored in a cache-efficient 16-byte/splat format.
|
||||
|
||||
## "Programmable Splats"
|
||||
|
||||
Forge also uses this opportunity to run a user-programmable data pipeline on each splat on the GPU. The standard pipeline provides high-level functionality, such as applying rigid transforms, adjusting RGB / opacity, and spherical harmonics, but also color editing and perturbations (via `SplatEdit`) and a dual-quaternion skeletal animation system (`SplatSkinning`). The standard pipeline also allows injecting arbitrary code to modify each splat via `dyno` shader graph system.
|
||||
Spark also uses this opportunity to run a user-programmable data pipeline on each splat on the GPU. The standard pipeline provides high-level functionality, such as applying rigid transforms, adjusting RGB / opacity, and spherical harmonics, but also color editing and perturbations (via `SplatEdit`) and a dual-quaternion skeletal animation system (`SplatSkinning`). The standard pipeline also allows injecting arbitrary code to modify each splat via `dyno` shader graph system.
|
||||
|
||||
`SplatMesh` derives from a more general base class `SplatGenerator`, which itself derives from `THREE.Object3D`. As such, it can be placed anywhere in the scene hierarchy and obeys expected local and global coordinate transforms. A `SplatGenerator` is the most general form of a "splat object", whose splats are produced programmatically via a `dyno` shader graph function that maps `{ index: "int" }` to `{ gsplat: "Gsplat" }`. A `SplatMesh` is a higher-level object that implements such a mapping, reading source splats from a template (loaded via a `url` constructor parameter or otherwise) at the given `index`, then applying functions such as transforming to world space.
|
||||
|
||||
In contrast, implementing a `SplatGenerator` gives you full control to write any function that programmatically computes a splat's attributes (center, scales, quaternion, rgba). These could be stateless (relying only on `index`, random-number generators, etc), or could rely on a complex combination of splat files, textures, and other global parameters for real-time procedural generation, and can vary with time to produce real-time animations.
|
||||
|
||||
The `dyno` shader graph system allows you to create these programmatic pipelines with Javascript code, which is synthesized into GLSL code and compiled and run on the GPU. This `dyno` system powers other components of Forge as well, such as `Readback` (which can perform any computation and read back the resulting value), used to compute the sort distance metric for pairs of splats and read them back for CPU sorting.
|
||||
The `dyno` shader graph system allows you to create these programmatic pipelines with Javascript code, which is synthesized into GLSL code and compiled and run on the GPU. This `dyno` system powers other components of Spark as well, such as `Readback` (which can perform any computation and read back the resulting value), used to compute the sort distance metric for pairs of splats and read them back for CPU sorting.
|
||||
@@ -24,7 +24,7 @@
|
||||
</svg>
|
||||
</a>
|
||||
</div>
|
||||
<a rel="noopener noreferrer" class="contrast" aria-label="GitHub repository" href="https://github.com/forge-gfx/forge" target="_blank" style="margin-top: 5px;">
|
||||
<a rel="noopener noreferrer" class="contrast" aria-label="GitHub repository" href="https://github.com/sparkjs-dev/spark" target="_blank" style="margin-top: 5px;">
|
||||
<svg xmlns="http://www.w3.org/2000/svg" height="16" width="16" viewBox="0 0 496 475" class="icon-github">
|
||||
<path d="M165.9 397.4c0 2-2.3 3.6-5.2 3.6-3.3.3-5.6-1.3-5.6-3.6 0-2 2.3-3.6 5.2-3.6 3-.3 5.6 1.3 5.6 3.6zm-31.1-4.5c-.7 2 1.3 4.3 4.3 4.9 2.6 1 5.6 0 6.2-2s-1.3-4.3-4.3-5.2c-2.6-.7-5.5.3-6.2 2.3zm44.2-1.7c-2.9.7-4.9 2.6-4.6 4.9.3 2 2.9 3.3 5.9 2.6 2.9-.7 4.9-2.6 4.6-4.6-.3-1.9-3-3.2-5.9-2.9zM244.8 8C106.1 8 0 113.3 0 252c0 110.9 69.8 205.8 169.5 239.2 12.8 2.3 17.3-5.6 17.3-12.1 0-6.2-.3-40.4-.3-61.4 0 0-70 15-84.7-29.8 0 0-11.4-29.1-27.8-36.6 0 0-22.9-15.7 1.6-15.4 0 0 24.9 2 38.6 25.8 21.9 38.6 58.6 27.5 72.9 20.9 2.3-16 8.8-27.1 16-33.7-55.9-6.2-112.3-14.3-112.3-110.5 0-27.5 7.6-41.3 23.6-58.9-2.6-6.5-11.1-33.3 2.6-67.9 20.9-6.5 69 27 69 27 20-5.6 41.5-8.5 62.8-8.5s42.8 2.9 62.8 8.5c0 0 48.1-33.6 69-27 13.7 34.7 5.2 61.4 2.6 67.9 16 17.7 25.8 31.5 25.8 58.9 0 96.5-58.9 104.2-114.8 110.5 9.2 7.9 17 22.9 17 46.4 0 33.7-.3 75.4-.3 83.6 0 6.5 4.6 14.4 17.3 12.1C428.2 457.8 496 362.9 496 252 496 113.3 383.5 8 244.8 8zM97.2 352.9c-1.3 1-1 3.3.7 5.2 1.6 1.6 3.9 2.3 5.2 1 1.3-1 1-3.3-.7-5.2-1.6-1.6-3.9-2.3-5.2-1zm-10.8-8.1c-.7 1.3.3 2.9 2.3 3.9 1.6 1 3.6.7 4.3-.7.7-1.3-.3-2.9-2.3-3.9-2-.6-3.6-.3-4.3.7zm32.4 35.6c-1.6 1.3-1 4.3 1.3 6.2 2.3 2.3 5.2 2.6 6.5 1 1.3-1.3.7-4.3-1.3-6.2-2.2-2.3-5.2-2.6-6.5-1zm-11.4-14.7c-1.6 1-1.6 3.6 0 5.9 1.6 2.3 4.3 3.3 5.6 2.3 1.6-1.3 1.6-3.9 0-6.2-1.4-2.3-4-3.3-5.6-2z"></path>
|
||||
</svg>
|
||||
|
||||
Reference in New Issue
Block a user