This PR Adds the ability to set the color of the skybox using the URL
parameters.
- Now you can pass a color's hexadecimal number (without the #) in the
skybox parameter, and that will set the skybox to that color.
This is a small feature push that adds a simple Shader for the Gizmo.
This shader only takes one value: the color of the object. Very simple
and fast to compile.
This PR is missing:
- Shader Pre Compiling. This is something that I am looking into now,
however I thought it out of scope for this feature.
Just thought I would drop this real fast. Let me know what y'all think!
---------
Co-authored-by: Danny McGee <dannymcgee@gmail.com>
This PR adds support for Mixamo FBX imports (via backend conversion to
glTF/glB) and some miscellaneous improvements to the animation systems.
### User-facing improvements
* Retargeting math has been fixed to eliminate jittering issues
* Retargeter can now handle animated bone translations when present
(generally this is only used for root motion)
### Housekeeping / Code improvements
* Animation-related components and system logic have been moved to a
dedicated `anim` module and `AnimPlugin`
* Retargeting stuff has been moved from the `bvh` module to
`anim::retargeting`
* Component-based solution for marking an entity as an animation
retargeting source makes it easy to extend the retargeting functionality
to new types of source animations
### Limitations
Currently the code assumes that there will be at most one animation
source at a time. That single animation (if present) will be retargeted
to all other skeletons in the scene. A component-based solution for
directing a specific target skeleton to use a specific source animation
is possible, but will require some additional consideration around the
implementation design for the read-only "viewer" modes.
### Test animations:
* Hip-Hop Dancing (Mixamo): ?mode=viewer&mixamo=mocap/hip-hop-dancing.gltf
* Shuffle (MocapNET): ?mode=viewer&bvh=mocap/shuffle.bvh
* Scott (MocapNET): ?mode=viewer&bvh=mocap/scott.bvh
* Scott 1 (MocapNET): ?mode=viewer&bvh=mocap/scott1.bvh
Just adding the mannequin from Scott into the main branch so that it can
be used.
- added the mannequin.gltf
- also now requiring the file extention when selecting file to load.
This is to add flexability with file selection. Let me know if this is a
bad idea.
This PR adds support for BVH animation files, with a custom animation
player and retargeting solution for our Blender Rigify-based skeletal
meshes.
---------
Co-authored-by: Brooks Palin <brookspalin710@gmail.com>
This commit implements a Sketchfab-style single-object viewer as a minimal
proof-of-concept.
### `@storyteller/studio-web`
This is a new package to house web components for interfacing with the
`@storyteller/studio` WASM library. It currently includes a single component,
`<sts-viewer />`, which is exported from `@storyteller/studio-web/viewer`.
### `react-frontend`
This is a new application intended as a proof-of-concept and testbed for React
interop with `@storyteller/studio-web`.
Web components are currently a little bit fiddly to use in a React application.
`react-frontend/src/app/studio-viewer.tsx` demonstrates how a React-idiomatic
component can be written as a thin wrapper around a web component, and
`react-frontend/src/app/app.tsx` demonstrates that wrapper component in use.
* Migrate `System::pipe(...)` calls to `System::map(...)`
* Migrate `EventReader::iter()` and `EventReader::into_iter()` calls to
`EventReader::read()`
* Migrate asset handling
* Generate `*.meta` files for assets
* Migrate a shader import
* Fix emissive materials double-applying their emissive strength
* Fix non-normalized bone weights resulting in incorrect vertex positions
* Set shadow filtering to `Hardware2x2` to mitigate increase in shader
compilation times
This PR adds functional transform gizmos, along with some new frontend UI to
configure their state.
## Work Done
- [x] Global / Local space toggling
- [x] Gizmo type toggling
- [x] Translation gizmo
- [x] Axis-constrained translation
- [x] Plane-constrained translation
- [x] Screen-space translation
- [x] Rotation gizmo
- [x] Axis-constrained rotation
- [x] Screen-space rotation
- [x] "Trackball" rotation
- [ ] Scale gizmo
- [ ] Single-axis non-uniform scaling
- [ ] Two-axis non-uniform scaling
- [ ] Uniform scaling
- [ ] Combination gizmos
- [x] Translation + Rotation
- [ ] Translation + Scale
- [ ] Rotation + Scale
- [ ] Translation + Rotation + Scale
## Future Enhancements
### Better visual feedback during transformations
Most DCCs update the visual representation of the transform gizmo while it's in
the process of being actively manipulated. (For example, Blender draws lines in
world-space to indicate that the transformation is constrained to a particular
axis or plane, hides widgets that aren't participating in the current
transformation, and illustrates the angle of difference between the original and
updated rotation.) At a minimum, we should indicate the current widget being
manipulated with a color change (like we already do when the widget is being
hovered). This requires a bit of additional state management effort but
shouldn't be overly difficult to add in a future PR.
### Better visual representation for rotation widgets
Currently, each axis of rotation is represented on the gizmo with a full-circle,
unlit "ring" shape. This can make it difficult to tell which half of the ring is
closer to the viewer, and thus to get an accurate understanding of how the
transform is currently oriented, without moving the camera around. A [previous
version of this work](#4) bisected the rings along a view-oriented plane and
rendered the "back-side" ring halves with reduced opacity. This PR uses static
meshes instead of an immediate-mode drawing API to render the gizmos, so a
custom shader would be the ideal way to reproduce that effect. That effort
should be deferred until after upgrading to Bevy 0.12, as it brings some changes
which would likely require some updates to that code.
### Better visual representation for translation arrows
A [previous version of this work](#4) faded the opacity of the translation
arrows when they were very close to parallel with the view direction — an idea
that was borrowed from Blender. It's a nice UX enhancement because translating
along an axis that's very close to parallel with the view direction can be janky
and error-prone, so it makes sense to disallow it and instead provide an
unobstructed view of the screen-space translation widget. This would also
require a custom shader, in addition to some extra state management logic (to
disable the widgets when their opacity is zeroed out). Given that it's only a
"nice-to-have," I think it makes sense to backlog this for the time being.
## Known Issues
### Picking
There's currently a bug in `bevy_mod_picking` that affects the hit-testing of
the gizmo widgets. The gizmos are currently _rendering_ on top of everything
else (via a second camera and render-layer configurations), but the picking
behavior doesn't take the rendering order into account. This means that portions
of the gizmo widgets which are occluded by geometry in the world cannot be
interacted with (because the occluding geometry interrupts the picking raycast).
> Update: This is fixed in the new release of `bevy_mod_picking` for Bevy 0.12.
### Delay on first-render
The first time a new widget shape is spawned, there's a noticeable delay (of 1
to several seconds) before the widget is rendered, during which the UI becomes
unresponsive. This is likely the same issue that triggers a much longer delay
when the glTF scene is being spawned. I'd like to see if Bevy 0.12 resolves or
mitigates this issue before addressing it on our end, but if it doesn't, I think
we can simply spawn an instance of each of the gizmo widgets off-screen on
application startup to prevent lag spikes from happening while a user is
interacting with the scene.
### Non-deterministic hover states
The widget colors are currently being updated by, e.g., multiplying on
mouser-over and dividing on mouse-out. Occasionally those events happen in quick
enough succession that our mouse-out listener is not invoked, so a widget gets
"stuck" in its hovered state. The fix for this is trivial but fairly annoying,
so I haven't gotten around to it yet. 🙂
> Update: This issue mostly occurs when in world-space mode. The gizmo is torn
> down and re-built from scratch once a drag event completes, which can despawn
> the gizmo while an element is still hovered. The solution will likely involve
> completely separate materials for default and hover states, similar to how
> `bevy_mod_picking`'s highlight plugin works.
### Object Selection
There's still some jank happening with object selection and camera orbiting, as
mentioned in #8. I don't anticipate this being overly difficult to resolve, but
I'm deferring that work until more of the core functionality is built out since
it's not a blocker for now.
### Skinned Mesh Selection
There's also still the issue of the skinned mesh being unselectable — which, it
turns out, is [technically not actually the
case](https://github.com/bevyengine/bevy/issues/4971). Rather, the picking
raycast "sees" the mesh as if it were still in its original rest pose, because
the mesh deformations are processed on the GPU and not reflected back to the
CPU-side representation.
I think the ideal workaround for this will be to attach capsule-shaped Rapier
colliders to the skeleton (similar to Unreal Engine's auto-generated "physics
assets" for skeletal meshes) and use those for picking instead. This would
automatically enable skeletal pose manipulation (by transforming the individual
joints), which is already a long term goal. It also makes more sense from a
practical standpoint, as the mesh itself cannot be independently transformed
while it's bound to the skeleton, so there's little point in directly selecting
it.
Brooks Palin