mirror of
https://github.com/storytold/spark.git
synced 2026-10-09 00:09:53 +00:00
Add dyno-overview and dyno-stdlib to docs. Update mkdocs. Minor fixes in docs. Remove unneeded imports in util.ts.
This commit is contained in:
Andreas Sundquist
committed by
Andreas Sundquist
Andreas Sundquist
parent
7f92eb2754
commit
6a5b361181
@@ -1,6 +1,6 @@
|
||||
# Controls
|
||||
|
||||
A program using `Forge` can use any camera control scheme that is compatible with Three.js and will typically manipulate a `THREE.Camera` object's transform. `Forge` also ships with simple, intuitive controls for navigating 3D space that use the keyboard + mouse, game pad, or mobile multi-touch. To add these controls, you can create a `ForgeControls` instance:
|
||||
A program using `Forge` can use any camera control scheme that is compatible with THREE.js and will typically manipulate a `THREE.Camera` object's transform. `Forge` also ships with simple, intuitive controls for navigating 3D space that use the keyboard + mouse, game pad, or mobile multi-touch. To add these controls, you can create a `ForgeControls` instance:
|
||||
|
||||
```typescript
|
||||
const controls = new ForgeControls({
|
||||
|
||||
@@ -0,0 +1,148 @@
|
||||
# Dyno shaders
|
||||
|
||||
The `dyno` shader graph system is one of the architectural pillars of Forge, allowing you to create custom computation graphs using Javascript (and optionally GLSL) that are compiled to GLSL and run on the GPU, similar to shader graph systems in modern 3D graphics engines.
|
||||
|
||||
A core component of this system is the class `Dyno` and its subclasses, which can be thought of as function blocks with multiple typed inputs and outputs. Values passed between such blocks are of type `DynoVal<T>`, where `T` must be a `DynoType`, representing a GPU type in GLSL. Using TypeScript, Forge ensures type safety and static validation of the GPU computation graph.
|
||||
|
||||
All `dyno` code is contained within `src/dyno`, and has definitions which cover all built-in GLSL ES 3.0 types (`"int"`, `"float"`, `"vec"`, etc.) and most of the standard functions (`mul`, `cross`, `texelFetch`, etc.). Note that regular Javascript functions can't be part of a `dyno` computation graph: instead of `x + y` you should use `dyno.add(x, y)`. You can also define your own custom types and `dyno` blocks, both by compositing existing `dyno` functions or by writing GLSL code directly.
|
||||
|
||||
Forge currently uses these Dynos in two main places:
|
||||
|
||||
- Dynamically generating splats from `SplatGenerator`/`SplatMesh` into the scene
|
||||
- Computing the splat distance metric for CPU readback and sorting
|
||||
|
||||
At the lowest layer, these are executed as pseudo-compute shaders by the classes `PackedSplats` and `Readback`, respectively. A `PackedSplats` is a collection of splats stored in a cache-efficient 16-byte/splat format, which can be loaded from a splat file URL, or generated via `dyno` program that maps an integer index `DynoVal<"int">` to a `DynoVal<Gsplat>`, a GLSL struct type that contains all the splat parameters. Similarly, `Readback` takes a `dyno` program that maps an index `DynoVal<"int">` to a RGBA8 value via a `DynoVal<"vec4">`, producing a 32-bit value per index that can then be read back to the CPU for splat sorting.
|
||||
|
||||
Learning to build and use `dyno` programs is probably best approached by starting with the Particle Simulation example and by examining `SplatMesh.constructGenerator()` for a more complex graph. `SplatMesh` also has two injection points `objectModifier` and `worldModifier` that allow you to inject `dyno` component into the standard splat generation pipeline to modify an existing splat object before it is rendered.
|
||||
|
||||
## `type DynoType`
|
||||
|
||||
A `DynoType` can be either a string that corresponds to a built-in GLSL type, or `{ type: "MyType" }` for a user-defined type. These types are used both for identifying value types `DynoVal<T extends DynoType>` and for declaring input/output types for `Dyno` blocks, for example `{ index: "int" }` or `{ gsplat: Gsplat }` (`Gsplat` is defined as `{ type: "Gsplat" }`). Forge uses these to enforce TypeScript constraints on inputs+outputs of `Dyno` blocks to generate correct GLSL code.
|
||||
|
||||
### Build-in types
|
||||
|
||||
A built-in GLSL type can be single-valued (`"int"`, `"uint"`, `"float"`, `"bool"`), vector (`"vec4"`, `"ivec3"`, `"bvec2"`), matrix (`"mat4"`, `"mat2x3"`), or a sampler (`"sampler2D"`, `"usampler3D"`, etc).
|
||||
|
||||
### Custom types
|
||||
|
||||
Forge defines a handful of custom types that are useful: `Gsplat`, `TPackedSplats`, `SdfArray`, `TRgbaArray`, and `SplatSkinning`. For example, in `src/dyno/splats.ts` we define `Gsplat` as `{ type: "Gsplat" }` along with a helper function:
|
||||
|
||||
```typescript
|
||||
export const defineGsplat = unindent(`
|
||||
struct Gsplat {
|
||||
vec3 center;
|
||||
uint flags;
|
||||
vec3 scales;
|
||||
int index;
|
||||
vec4 quaternion;
|
||||
vec4 rgba;
|
||||
};
|
||||
const uint GSPLAT_FLAG_ACTIVE = 1u << 0u;
|
||||
|
||||
bool isGsplatActive(uint flags) {
|
||||
return (flags & GSPLAT_FLAG_ACTIVE) != 0u;
|
||||
}
|
||||
`);
|
||||
```
|
||||
|
||||
When declaring a new user type, you should also define how that type will appear in GLSL, along with any other definitions or utility functions that are useful for that type. When creating a new `Dyno` block you can then include these definitions wherever a `Gsplat` type is used to make sure these global definitions are available.
|
||||
|
||||
## `type DynoVal<T extends DynoType>`
|
||||
|
||||
A `DynoVal<T extends DynoType>` corresponds to a value of type `T` in the GLSL computation graph. These values can come from a literal/constant value, a uniform (a value that can be updated every frame from your program), or the output of a `Dyno` block.
|
||||
|
||||
### `DynoLiteral` / `dynoLiteral`
|
||||
|
||||
The class `DynoLiteral<T extends DynoType>` is a `DynoVal` whose constant value is given as a GLSL literal string. For example, `new DynoLiteral("float", "1.0")` or `new DynoLiteral("vec3", "vec3(1.0, 2.0, 3.0)")` produce `DynoVal`s that can be used as inputs in a `dyno` graph.
|
||||
|
||||
The helper function `dynoLiteral(type, literal)` is a shorthand for `new DynoLiteral(type, literal)`.
|
||||
|
||||
### `DynoConst` / `dynoConst`
|
||||
|
||||
Creating a new `DynoConst<T extends DynoType>` is similar to a `DynoLiteral`, but the value is given as a Javascript value, which is converted to a GLSL literal string. For example, `new DynoConst("float", 1.0)` or `new DynoConst("vec3", new THREE.Vector3(1.0, 2.0, 3.0))` produce `DynoVal`s that can be used as inputs in a `dyno` graph.
|
||||
|
||||
You can also use the helper function `dynoConst(type, value)` to create a `DynoConst`.
|
||||
|
||||
### `DynoOutput`
|
||||
|
||||
A `DynoOutput` is created internally in the `dyno` system to represent a particular named output of a `Dyno` block. For example, `dyno.splitGsplat` takes a `DynoVal<Gsplat>` as input and produces multiple outputs including `index`, `center`, `rgba`, etc., and selecting a particular output will yield the appropriate typed `DynoOutput`:
|
||||
|
||||
```typescript
|
||||
// opacity is a DynoOutput and also a DynoVal<"float">
|
||||
const { opacity } = dyno.splitGsplat(gsplat);
|
||||
```
|
||||
|
||||
## `class Dyno<InTypes, OutTypes>`
|
||||
|
||||
A `Dyno` an abstract block that has named, typed inputs and outputs, both as part of its type signature (for TypeScript checking) and stored in the class (for runtime access). When creating a new `Dyno` the following constructor options are available:
|
||||
|
||||
| **Option** | **Description** |
|
||||
|----------|-------------|
|
||||
| `inTypes` | A map from input name to `DynoType`, for example `{ index: "int" }`. This must match the template parameter `InTypes` of the `Dyno` class. |
|
||||
| `outTypes` | A map from output name to `DynoType`, for example `{ rgba: "vec4" }`. This must match the template parameter `OutTypes` of the `Dyno` class. |
|
||||
| `inputs` | A mapping of input values that are passed to the `Dyno`, where the type of `DynoVal<T>` must match the `DynoType` in `inTypes`. |
|
||||
| `update` | Optional function that is called (no return value) each time a program containing this `Dyno` is executed. This can for example be used to update any uniforms before running. |
|
||||
| `globals` | Optional function that outputs global GLSL definitions needed by this block, as an array of strings. Duplicate global definition strings within the same program will be deduplicated, so you should always define all the globals you need for this block without regard to whether they are defined elsewhere. |
|
||||
| `statements` | Optional function that outputs the GLSL statements to execute for this block as an array of strings. When defining a custom `Dyno` that executes GLSL, this is where your code will go. |
|
||||
| `generate` | Called internally by the `dyno` compiler to generate the globals, statements, and any uniforms needed for this block. The default `generate` implementation will call `globals` and `statements` if they are defined and return these to the compiler. |
|
||||
|
||||
The methods `globals`, `statements`, and `generate` all take a `GenerateContext` object as argument, which consists of three properties:
|
||||
|
||||
| **Property** | **Description** |
|
||||
|----------|-------------|
|
||||
| `inputs` | A map from the input key to the *actual* GLSL variable name of that input value. The compiler will create unique names for values passed between blocks so you can reuse the same input key. If an input is NOT attached, the value will be `undefined`. |
|
||||
| `outputs` | A map from the output key to the *actual* GLSL variable name of that output value. The compiler will create unique names for values passed between blocks so you can reuse the same output key. If an output is NOT connected, the value will be `undefined`. |
|
||||
| `compile` | An instance of class `Compilation` that stores the context for the ongoing compilation, including global defines, statements to execute, uniforms, indentation level, etc. |
|
||||
|
||||
Typically you will want to define a `Dyno` subclass that constructs a `Dyno` instance with the appropriate `globals`, `statements`, and `update` etc. for your block. When generating code, use the passed in `GenerateContext` to get the names of inputs and outputs, and generate code that uses those names (via string interpolation or similar).
|
||||
|
||||
### Using a `Dyno`
|
||||
|
||||
Once defined, using your `Dyno` will typically be done as follows:
|
||||
```typescript
|
||||
const { sum } = new dyno.Add({ a, b }).outputs;
|
||||
// Or equivalently:
|
||||
const { sum } = dyno.add(a, b).outputs;
|
||||
// For Dynos that implement HasDynoOut<T> you can also do
|
||||
const sum = dyno.add(a, b);
|
||||
```
|
||||
|
||||
The property `outputs` is a map from output name to a `DynoVal<T>` for each output type in `outTypes` (internally they are `DynoOutput`s).
|
||||
|
||||
For a `Dyno` that only has a single output, you can also implement the method `dynoOut(): DynoValue<T>` (which corresponds to the interface `HasDynoOut<T>`) to pick out a default output from `outputs`, which can help reduce redundant `.outputs` calls.
|
||||
|
||||
### Helper functions for GLSL code
|
||||
|
||||
`unindentLines(s: string): string[]`: Takes a multi-line string, removes common leading whitespace from each line, and returns the resulting array of strings. This can be useful to prepare an array of `statements` from a block of GLSL code:
|
||||
```typescript
|
||||
const leftAlignedLines = unindentLines(`
|
||||
float sqr(float x) {
|
||||
return x * x;
|
||||
}
|
||||
`);
|
||||
```
|
||||
|
||||
`unindent(s: string): string`: Same as for `unindentLines` but joins the resulting lines into a single string. This is useful for defining a `globals`, which are de-duplicated based on the entirety of the string rather than individual lines.
|
||||
|
||||
## `class DynoBlock<InTypes, OutTypes>`
|
||||
|
||||
A `DynoBlock` is a `Dyno` that is like a "module" in that it can contain a `Dyno` subgraph internally that is created in a closure. Although you can construct a `DynoBlock` directly it is more ergnomic to use the helper function `dynoBlock`:
|
||||
|
||||
```typescript
|
||||
const myGsplatGenerator = dyno.dynoBlock(
|
||||
{ index: "int" }, // Inputs
|
||||
{ gsplat: Gsplat }, // Outputs
|
||||
({ index }) => { // Closure mapping inputs to outputs
|
||||
let gsplat = dyno.readPackedSplat(myPackedSplats, index);
|
||||
const opacity = dyno.dynoConst("float", 1.0);
|
||||
gsplat = dyno.combineGsplat({ gsplat, opacity });
|
||||
return { gsplat };
|
||||
},
|
||||
);
|
||||
```
|
||||
|
||||
The first argument is mapping from input keys to `DynoType`, the second for output keys, and the third argument is a closure that constructs a `dyno` graph from the input `DynoVal`s and any other external `DynoVal`s (such as `myPackedSplats` in this example), and must return a map of `DynoVal`s that match the output keys. An optional fourth argument allows you to pass in additional arguments `update` (called before program execution) and `globals` (for any global definitions needed by the block).
|
||||
|
||||
## Helpers to design Dynos
|
||||
|
||||
When creating blocks that are simple operations of 1-3 inputs you can use the helper classes `UnaryOp`, `BinaryOp`, and `TrinaryOp`, which help standardize the interface. You must override the class' `statements` or `generate` method to generate the code. See examples in `src/dyno/math.ts` for how to use them.
|
||||
@@ -0,0 +1,235 @@
|
||||
# Dyno Standard Library
|
||||
|
||||
The Forge `dyno` system provides a standard library of `Dyno` blocks that cover most of the built-in functions in GLSL ES 3.0, including data conversion, logic, math, trigonometry, linear algebra, texture lookups, transforms, managing uniform variables, hashing & RNG, and of course managing splat data.
|
||||
|
||||
We use the convention of PascalCase for the names of the `Dyno` classes, and camelCase for the names of equivalent helper functions that are more ergonomic to use. For example, you can equivalently write:
|
||||
```typescript
|
||||
const sum = new dyno.Add({ a, b });
|
||||
const sum = dyno.add(a, b);
|
||||
```
|
||||
|
||||
The second form will create the `dyno.Add` class with appropriate constructor options. In the tables below we will opt for the camelCase variant for brevity.
|
||||
|
||||
## Data type conversion
|
||||
|
||||
The following functions use standard GLSL casting rules, which means that for example `bool(1)` will return `true` and `bool(0)` will return `false`.
|
||||
|
||||
| **Function** | **Description** |
|
||||
|----------|-------------|
|
||||
| `bool(value): DynoVal<"bool">` | Convert a value to a boolean |
|
||||
| `int(value): DynoVal<"int">` | Convert a value to a signed integer |
|
||||
| `uint(value): DynoVal<"uint">` | Convert a value to an unsigned integer |
|
||||
| `float(value): DynoVal<"float">` | Convert a value to a floating-point number |
|
||||
| `bvec<N>(value): DynoVal<"bvec<N>">` | Convert a value to a boolean vector of length N |
|
||||
| `ivec<N>(value): DynoVal<"ivec<N>">` | Convert a value to a signed integer vector of length N |
|
||||
| `uvec<N>(value): DynoVal<"uvec<N>">` | Convert a value to an unsigned integer vector of length N |
|
||||
| `vec<N>(value): DynoVal<"vec<N>">` | Convert a value to a floating-point vector of length N |
|
||||
| `mat<N>(value): DynoVal<"mat<M>">` | Convert a value to a NxN matrix |
|
||||
| `floatBitsToInt(value: DynoVal<"float">): DynoVal<"int">` | Reinterpret the bits of a float as an integer |
|
||||
| `floatBitsToUint(value: DynoVal<"float">): DynoVal<"uint">` | Reinterpret the bits of a float as an unsigned integer |
|
||||
| `intBitsToFloat(value: DynoVal<"int">): DynoVal<"float">` | Reinterpret the bits of an integer as a float |
|
||||
| `uintBitsToFloat(value: DynoVal<"uint">): DynoVal<"float">` | Reinterpret the bits of an unsigned integer as a float |
|
||||
| `packSnorm2x16(value: DynoVal<"vec2">): DynoVal<"uint">` | Encode a vec2 from -1..1 as a 16-bit signed integer and pack into a 32-bit uint |
|
||||
| `unpackSnorm2x16(value: DynoVal<"uint">): DynoVal<"vec2">` | Decode a 32-bit uint as a vec2 from -1..1 |
|
||||
| `packUnorm2x16(value: DynoVal<"vec2">): DynoVal<"uint">` | Encode a vec2 from 0..1 as a 16-bit unsigned integer and pack into a 32-bit uint |
|
||||
| `unpackUnorm2x16(value: DynoVal<"uint">): DynoVal<"vec2">` | Decode a 32-bit uint as a vec2 from 0..1 |
|
||||
| `packHalf2x16(value: DynoVal<"vec2">): DynoVal<"uint">` | Encode a vec2 as two float16 values and pack into a 32-bit uint |
|
||||
| `unpackHalf2x16(value: DynoVal<"uint">): DynoVal<"vec2">` | Decode a 32-bit uint as two float16 values |
|
||||
| `uintToRgba8(value: DynoVal<"uint">): DynoVal<"vec4">` | Decode a 32-bit uint as a vec4 of 8-bit unsigned integers |
|
||||
|
||||
## Logic
|
||||
|
||||
The following logic and bit operations follow standard GLSL ES 3.0 semantics.
|
||||
|
||||
| **Function** | **Description** |
|
||||
|----------|-------------|
|
||||
| `and(a, b)` | Logical (for bool output) or bit-wise (for integer output) AND |
|
||||
| `or(a, b)` | Logical (for bool output) or bit-wise (for integer output) OR |
|
||||
| `xor(a, b)` | Logical (for bool output) or bit-wise (for integer output) XOR |
|
||||
| `not(a)` | Logical NOT (for bool output) or bit-wise NOT (for integer output) |
|
||||
| `lessThan(a, b)` | Less than |
|
||||
| `lessThanEqual(a, b)` | Less than or equal to |
|
||||
| `greaterThan(a, b)` | Greater than |
|
||||
| `greaterThanEqual(a, b)` | Greater than or equal to |
|
||||
| `equal(a, b)` | Equal |
|
||||
| `notEqual(a, b)` | Not equal |
|
||||
| `any(a: DynoVal<"bvec<N>">)` | True if any component of the vector is true |
|
||||
| `all(a: DynoVal<"bvec<N>">)` | True if all components of the vector are true |
|
||||
| `select(cond, t: DynoVal<T>, f: DynoVal<T>): DynoVal<T>` | Select between two values based on a condition |
|
||||
| `compXor(a)` | Component-wise XOR of a boolean or integer vector |
|
||||
|
||||
## Math
|
||||
|
||||
The following math functions follow standard GLSL ES 3.0 semantics, for example rules around multiplication of vector and matrix types.
|
||||
|
||||
| **Function** | **Description** |
|
||||
|----------|-------------|
|
||||
| `add(a, b)` | Addition |
|
||||
| `sub(a, b)` | Subtraction |
|
||||
| `mul(a, b)` | Multiplication |
|
||||
| `div(a, b)` | Division |
|
||||
| `imod(a, b)` | Integer modulus |
|
||||
| `mod(a, b)` | Floating-point modulus |
|
||||
| `modf(a)` | Seperate a floating-point number into its fractional and integer parts |
|
||||
| `neg(a)` | Negation |
|
||||
| `abs(a)` | Absolute value |
|
||||
| `sign(a)` | Sign of a number |
|
||||
| `floor(a)` | Floor of a floating-point number |
|
||||
| `ceil(a)` | Ceiling of a floating-point number |
|
||||
| `trunc(a)` | Truncate a floating-point number toward negative infinity |
|
||||
| `round(a)` | Round a floating-point number to the nearest integer |
|
||||
| `fract(a)` | Fractional part of a floating-point number |
|
||||
| `pow(a, b)` | a ^ b |
|
||||
| `exp(a)` | e ^ a |
|
||||
| `exp2(a)` | 2 ^ a |
|
||||
| `log(a)` | Natural logarithm of a |
|
||||
| `log2(a)` | Base-2 logarithm of a |
|
||||
| `sqr(a)` | a * a |
|
||||
| `sqrt(a)` | Square root of a |
|
||||
| `inversesqrt(a)` | 1 / sqrt(a) |
|
||||
| `min(a, b)` | Minimum of two values |
|
||||
| `max(a, b)` | Maximum of two values |
|
||||
| `clamp(a, min, max)` | Clamp a value between two others |
|
||||
| `mix(a, b, t)` | Linear interpolation between two values |
|
||||
| `step(edge, x)` | 0 if x < edge, 1 otherwise |
|
||||
| `smoothstep(edge0, edge1, x)` | 0 if x <= edge0, 1 if x >= edge1, otherwise a smooth Hermite interpolation between 0 and 1 |
|
||||
| `isNan(a)` | True if a is NaN |
|
||||
| `isInf(a)` | True if a is infinite |
|
||||
|
||||
## Trigonometry
|
||||
|
||||
| **Function** | **Description** |
|
||||
|----------|-------------|
|
||||
| `radians(degrees)` | Convert degrees to radians |
|
||||
| `degrees(radians)` | Convert radians to degrees |
|
||||
| `sin(x)` | Sine of x |
|
||||
| `cos(x)` | Cosine of x |
|
||||
| `tan(x)` | Tangent of x |
|
||||
| `asin(x)` | Arcsine of x |
|
||||
| `acos(x)` | Arccosine of x |
|
||||
| `atan(x)` | Arctangent of x |
|
||||
| `atan2(y, x)` | Arctangent of y/x |
|
||||
| `sinh(x)` | Hyperbolic sine of x |
|
||||
| `cosh(x)` | Hyperbolic cosine of x |
|
||||
| `tanh(x)` | Hyperbolic tangent of x |
|
||||
| `asinh(x)` | Inverse hyperbolic sine of x |
|
||||
| `acosh(x)` | Inverse hyperbolic cosine of x |
|
||||
| `atanh(x)` | Inverse hyperbolic tangent of x |
|
||||
|
||||
## Linear Algebra
|
||||
|
||||
| **Function** | **Description** |
|
||||
|----------|-------------|
|
||||
| `length(a)` | Length of a vector |
|
||||
| `distance(a, b)` | Distance between two vectors |
|
||||
| `dot(a, b)` | Dot product of two vectors |
|
||||
| `cross(a: DynoVal<"vec3">, b: DynoVal<"vec3">)` | Cross product of two 3-dimensional vectors |
|
||||
| `normalize(a)` | Normalize a vector |
|
||||
| `faceforward(a, b, c)` | Returns `a` if `dot(c, b) < 0`, otherwise returns `-a` |
|
||||
| `reflectVec(incident, normal)` | Reflect a vector around a normal |
|
||||
| `refractVec(incident, normal, eta)` | Refract a vector around a normal given an index of refraction |
|
||||
| `split(a)` | Split a vector into its components |
|
||||
| `combine(a)` | Create a vector from components, or inject components into an existing vector |
|
||||
| `projectH(a)` | Project a vector in homogeneous coordinates |
|
||||
| `extendVec(a, b)` | Extend a vector with an additional component |
|
||||
| `swizzle(a, select)` | Select a subset of components from a vector |
|
||||
| `compMult(a, b)` | Component-wise multiplication of two vectors |
|
||||
| `outer(a, b)` | Outer product of two vectors |
|
||||
| `transpose(a)` | Transpose a matrix |
|
||||
| `determinant(a)` | Determinant of a matrix |
|
||||
| `inverse(a)` | Invert a matrix |
|
||||
|
||||
## Texture lookups
|
||||
|
||||
| **Function** | **Description** |
|
||||
|----------|-------------|
|
||||
| `textureSize(texture, lod?)` | Return the size of a texture |
|
||||
| `texture(texture, coord, bias?)` | Sample a texture at a continuous coordinate |
|
||||
| `texelFetch(texture, coord, lod?)` | Fetch a discrete texel value from a texture |
|
||||
|
||||
## Transforms
|
||||
|
||||
| **Function** | **Description** |
|
||||
|----------|-------------|
|
||||
| `transformPos(position, { scale?, scales?, rotate?, translate? })` | Performs a transform of a position in 3D space, with optional uniform `scale`, anisotropic `scales`, quaternion `rotate`, and `translate` |
|
||||
| `transformDir(dir, { scale?, scales?, rotate? })` | Performs a transform of a direction in 3D space, with optional uniform `scale`, anisotropic `scales`, and quaternion `rotate` |
|
||||
| `transformQuat(quat, { rotate? })` | Rotate a quaternion |
|
||||
|
||||
## Uniform variables
|
||||
|
||||
Constant values and literals in `dyno` programs should not be changed often because it incurs a recompilation. To have a variable that can be changed every frame, you can declare a "uniform". The base class for uniforms provided by Forge is `DynoUniform`, which importantly contains a type, current value, and update function.
|
||||
|
||||
To update a uniform, simply assign a new value to the `value` property of the uniform. Alternatively, you can construct a `DynoUniform` with an `update` function that is called for each execution. This function can either update `value` directly, or return any non-`undefined` value to have it updated.
|
||||
|
||||
Use the following helper functions for more ergonomic creation of uniforms.
|
||||
|
||||
| **Function** | **Description** |
|
||||
|----------|-------------|
|
||||
| `dynoBool(value)` | Create a boolean uniform |
|
||||
| `dynoUint(value)` | Create an unsigned integer uniform |
|
||||
| `dynoInt(value)` | Create a signed integer uniform |
|
||||
| `dynoFloat(value)` | Create a floating-point uniform |
|
||||
| `dynoBvec<N>(value)` | Create a boolean vector of length N |
|
||||
| `dynoUvec<N>(value)` | Create an unsigned integer vector of length N |
|
||||
| `dynoIvec<N>(value)` | Create a signed integer vector of length N |
|
||||
| `dynoVec<N>(value)` | Create a floating-point vector of length N |
|
||||
| `dynoMat<N>(value)` | Create an NxN matrix |
|
||||
| `dynoMat<N>x<M>(value)` | Create an NxM matrix |
|
||||
| `dynoUsampler2D(texture)` | Create a uniform that lets you sample a 2D uint texture |
|
||||
| `dynoIsampler2D(texture)` | Create a uniform that lets you sample a 2D int texture |
|
||||
| `dynoSampler2D(texture)` | Create a uniform that lets you sample a 2D float texture |
|
||||
| `dynoUsampler2DArray(texture)` | Create a uniform that lets you sample a 2D uint texture array |
|
||||
| `dynoIsampler2DArray(texture)` | Create a uniform that lets you sample a 2D int texture array |
|
||||
| `dynoSampler2DArray(texture)` | Create a uniform that lets you sample a 2D float texture array |
|
||||
| `dynoUsampler3D(texture)` | Create a uniform that lets you sample a 3D uint texture |
|
||||
| `dynoIsampler3D(texture)` | Create a uniform that lets you sample a 3D int texture |
|
||||
| `dynoSampler3D(texture)` | Create a uniform that lets you sample a 3D float texture |
|
||||
| `dynoUsamplerCube(texture)` | Create a uniform that lets you sample a cube uint texture |
|
||||
| `dynoIsamplerCube(texture)` | Create a uniform that lets you sample a cube int texture |
|
||||
| `dynoSamplerCube(texture)` | Create a uniform that lets you sample a cube float texture |
|
||||
| `dynoSampler2DShadow(texture)` | Create a uniform that lets you sample a 2D float shadow texture |
|
||||
| `dynoSampler2DArrayShadow(texture)` | Create a uniform that lets you sample a 2D float shadow texture array |
|
||||
| `dynoSamplerCubeShadow(texture)` | Create a uniform that lets you sample a cube float shadow texture |
|
||||
|
||||
## Hashing & Random number generation
|
||||
|
||||
When a `dyno` program executes, each invocation for a given splat/index is effectively run in parallel, separate from the rest. In order to incorporate randomness into a `dyno` program, you must use the inputs available to the program, which is often just the `index` of the splat itself. Forge provides functions to hash any scalar or vector (integer or float) into 1-4 components of either a uint32 or float, using the PCG random number generator.
|
||||
|
||||
| **Function** | **Description** |
|
||||
|----------|-------------|
|
||||
| `pcgMix(value): DynoVal<"uint">` | Mix scalar or vector values into a single uint32 value that can be used as a seed for PCG |
|
||||
| `pcgNext(state): DynoVal<"uint">` | Advance the PCG random number generator by one step |
|
||||
| `pcgHash(state): DynoVal<"uint">` | Hash the PCG state into a random uint32 |
|
||||
| `hash(value): DynoVal<"uint">` | Hash a scalar or vector value into a random uint32 |
|
||||
| `hash2(value): DynoVal<"uvec2">` | Hash a scalar or vector value into a random uvec2 |
|
||||
| `hash3(value): DynoVal<"uvec3">` | Hash a scalar or vector value into a random uvec3 |
|
||||
| `hash4(value): DynoVal<"uvec4">` | Hash a scalar or vector value into a random uvec4 |
|
||||
| `hashFloat(value): DynoVal<"float">` | Hash a scalar or vector value into a random float |
|
||||
| `hashVec2(value): DynoVal<"vec2">` | Hash a scalar or vector value into a random vec2 |
|
||||
| `hashVec3(value): DynoVal<"vec3">` | Hash a scalar or vector value into a random vec3 |
|
||||
| `hashVec4(value): DynoVal<"vec4">` | Hash a scalar or vector value into a random vec4 |
|
||||
|
||||
## Splat data
|
||||
|
||||
Forge makes it easier to work with splat data by defining the GLSL struct `Gsplat` which contains the following fields:
|
||||
|
||||
| **Field** | **Type** | **Description** |
|
||||
|----------|-------------|-------------|
|
||||
| `center` | `vec3` | Center of the splat |
|
||||
| `flags` | `uint` | Flags for the splat (0x1 = active) |
|
||||
| `scales` | `vec3` | Scales of the splat |
|
||||
| `index` | `int` | Index of the splat in the array |
|
||||
| `quaternion` | `vec4` | Quaternion orientation of the splat |
|
||||
| `rgba` | `vec4` | RGBA color of the splat |
|
||||
|
||||
This way, you can pass around a `DynoVal<Gsplat>` that contains the all the properties of a splat. The following helper functions are provided to extract the splat data from a `PackedSplats`:
|
||||
|
||||
| **Function** | **Description** |
|
||||
|----------|-------------|
|
||||
| `numPackedSplats(packedSplats)` | Get the number of splats in a `PackedSplats` |
|
||||
| `readPackedSplat(packedSplats, index)` | Read a particular splat from a `PackedSplats` by index |
|
||||
| `readPackedSplatRange(packedSplats, index, base, count)` | Read a particular splat from a `PackedSplats` by index but restricted to the given range |
|
||||
| `splitGsplat(gsplat)` | Split a `Gsplat` into its components |
|
||||
| `combineGsplat(gsplat)` | Create a `Gsplat` from components (or inject components into an existing `Gsplat`) |
|
||||
| `gsplatNormal(gsplat)` | Get the `Gsplat` normal, defined as whichever X/Y/Z axis has the smallest scale |
|
||||
| `transformGsplat(gsplat, { scale?, rotate?, translate?, recolor? })` | Transform a `Gsplat` and all its components by the given transform |
|
||||
@@ -10,7 +10,7 @@ The web's most popular 3D graphics library THREE.js, can't render 3DGS directly.
|
||||
|
||||
Forge is a dynamic 3DGS renderer built for THREE.js and WebGL2 that runs in any web browser (desktop, mobile, and WebXR). With a handful of lines of code, anyone using THREE.js can easily add 3DGS to their scenes (even by vibe coding!). By creating one or more `SplatMesh` objects and adding them to your scene, Forge will render these alongside traditional triangle-based meshes during your regular `render(scene, camera)` call. `SplatMesh` derives from `THREE.Object3D` and can be translated and rotated like any other object, placed arbitrarily in the scene hierarchy, and animated by adjusting the values each frame. A `SplatMesh` can be created from most splat file formats via the `url` parameter or by directly creating the splats one by one.
|
||||
|
||||
3DGS is still in its infancy, and we expect new techniques will be developed for recoloring/relighting, animation, transitions, and other creative or interactive effects. We designed Forge to be a programmable splat engine from the ground up, giving you unprecedented control over how individual splats are generated, animated, and rendered into the scene. Similar to shader graph systems in modern 3D graphics engines, Forge allows you to compose blocks of functions (called `Dyno`s) into computation graphs that can generate splats procedurally, modify them arbirarily, or anything other computation you can imagine, and will be converted to GLSL to run on the GPU.
|
||||
3DGS is still in its infancy, and we expect new techniques will be developed for recoloring/relighting, animation, transitions, and other creative or interactive effects. We designed Forge to be a programmable splat engine from the ground up, giving you unprecedented control over how individual splats are generated, animated, and rendered into the scene. Similar to shader graph systems in modern 3D graphics engines, Forge allows you to compose blocks of functions (called `Dyno`s) into computation graphs that can generate splats procedurally, modify them arbitrarily, or anything other computation you can imagine, and will be converted to GLSL to run on the GPU.
|
||||
|
||||
## Features
|
||||
- Integrates with THREE.js rendering pipeline to fuse splat and mesh-based objects
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# SplatMesh
|
||||
|
||||
A `SplatMesh` is a high-level interface for displaying and manipulating a "Splat mesh", a collection of Gaussian splats that serves as an "object" of sorts. It is analagous to a traditional triangle-based `THREE.Mesh`, which consists of geometry (points and triangles) and materials (color and lighting). Similarly, a `SplatMesh` contains geometry (splat centers, quaternion, and xyz scales) and materials (RGB color, opacity, spherical harmonics up to degree 3 for directional lighting), and can be added anywhere in the scene hierarchy.
|
||||
A `SplatMesh` is a high-level interface for displaying and manipulating a "Splat mesh", a collection of Gaussian splats that serves as an object of sorts. It is analagous to a traditional triangle-based `THREE.Mesh`, which consists of geometry (points and triangles) and materials (color and lighting). Similarly, a `SplatMesh` contains geometry (splat centers, orientation, and xyz scales) and materials (RGB color, opacity, spherical harmonics), and can be added anywhere in the scene hierarchy.
|
||||
|
||||
The usual THREE.js properties `position`, `quaternion`, `rotation` behave as you would expect, however `scale` only allows uniform scaling and averages the x/y/z scales. Additional properties `recolor` and `opacity` are multiplied in with the final splat color and opacity.
|
||||
|
||||
@@ -36,7 +36,7 @@ const splats = new SplatMesh({
|
||||
objectModifier?: GsplatModifier;
|
||||
worldModifier?: GsplatModifier;
|
||||
});
|
||||
// Add to scene to show splats (requires ForgeRenderer as well)
|
||||
// Add to scene to show splats
|
||||
scene.add(splats);
|
||||
```
|
||||
|
||||
@@ -83,7 +83,7 @@ Additional properties on a `SplatMesh` instance:
|
||||
| **skinning** | Optional `SplatSkinning` instance for animating splats with dual-quaternion skeletal animation. (default: `null`)
|
||||
| **edits** | Optional list of `SplatEdit`s to apply to the mesh. If `null`, any `SplatEdit` children in the scene graph will be added automatically. (default: `null`)
|
||||
| **splatRgba** | Optional `RgbaArray` to overwrite splat RGBA values with custom values. Useful for "baking" RGB and opacity edits into the `SplatMesh`. (default: `null`)
|
||||
| **maxSh** | Maximum Spherical Harmonics level to use. Call `updateGenerator()` after changing. (default: `3`)
|
||||
| **maxSh** | Maximum Spherical Harmonics level to use. Forge supports up to SH3. Call `updateGenerator()` after changing. (default: `3`)
|
||||
|
||||
## `dispose()`
|
||||
|
||||
|
||||
@@ -19,6 +19,8 @@ nav:
|
||||
- Loading Gsplats: docs/loading-splats.md
|
||||
- Procedural Splats: docs/procedural-splats.md
|
||||
- Splat RGBA-XYZ SDF editing: docs/splat-editing.md
|
||||
- Dyno overview: docs/dyno-overview.md
|
||||
- Dyno standard library: docs/dyno-stdlib.md
|
||||
- Controls: docs/controls.md
|
||||
- Performance tuning: docs/performance.md
|
||||
- Community Resources: docs/community-resources.md
|
||||
|
||||
+1
-14
@@ -1,20 +1,7 @@
|
||||
import { Dyno, DynoBlock, unindent } from "./base";
|
||||
import { float, vec2, vec3, vec4 } from "./convert";
|
||||
import { mul } from "./math";
|
||||
import {
|
||||
ScalarTypes,
|
||||
type ValueTypes,
|
||||
Vector2Types,
|
||||
Vector3Types,
|
||||
Vector4Types,
|
||||
isIntType,
|
||||
isScalarType,
|
||||
isUintType,
|
||||
isVector2Type,
|
||||
isVector3Type,
|
||||
isVector4Type,
|
||||
sameSizeUvec,
|
||||
} from "./types";
|
||||
import { type ValueTypes, isIntType, isUintType, sameSizeUvec } from "./types";
|
||||
import {
|
||||
DynoOutput,
|
||||
type DynoVal,
|
||||
|
||||
Reference in New Issue
Block a user