This PR begins scaffolding the Stable Diffusion Turbo application (title
TBD), which is a separate frontend for Storyteller Engine which focuses
on image generation.
### Design Overview
* The application renders a "split-screen" view, with Storyteller Studio
on the left, and the Stable Diffusion Turbo output on the right
* When changes are made in the Studio editor:
- A screenshot is taken from the Bevy application
- Screenshot is piped into Stable Diffusion Turbo model
- Model output is rendered to the screen
* This all happens in as close to real-time as possible
### Work completed in this PR
* The `studio-frontend` project has been (somewhat crudely) refactored
to split most of its components into reusable libraries
- `studio-ui` is a use-case-agnostic design system library for core UI
components. These are "dumb" components which are controlled
via `@property` inputs and send `CustomEvent`s in response to user
input.
- `studio-frontend` is a collection of reusable Storyteller Studio
widgets. These are "smart" components which depend on
`@storyteller/studio` directly, manage their own state, and require very
little configuration or styling. They are designed to be drop-in
building blocks that can be reused between various frontends.
* The new application has been bootstrapped as `image-generator-demo`
- Currently, this simply grabs screenshots from the Bevy application and
renders those screenshots on the right side of the split-screen view. A
future PR will wire up to the Storyteller.ai backend to pipe these
screenshots through Stable Diffusion Turbo, and render that model's
output instead.
### Testing
* Install dependencies as described in the repo's README
* Run `npx nx serve image-generator-demo`
* Open [localhost:4200](http://localhost:4200) in your browser
- Use the camera controller in the left side of the view to manipulate
the Bevy scene
- Verify the right side of the view updates with a fresh screenshot at a
rate of ~5 fps
* Toggle the "Realtime Capture" checkbox
- Verify the right side view no longer updates while the user is
actively manipulating the editor
- This should allow the app to run smoother on lower-spec hardware
---------
Co-authored-by: Danny McGee <dannymcgee@gmail.com>
* Migrate `System::pipe(...)` calls to `System::map(...)`
* Migrate `EventReader::iter()` and `EventReader::into_iter()` calls to
`EventReader::read()`
* Migrate asset handling
* Generate `*.meta` files for assets
* Migrate a shader import
* Fix emissive materials double-applying their emissive strength
* Fix non-normalized bone weights resulting in incorrect vertex positions
* Set shadow filtering to `Hardware2x2` to mitigate increase in shader
compilation times
This PR implements basic object selection, with help from `bevy_mod_picking` and
`bevy_mod_outline`.
### Work Done
* Click to select objects in the world
* Selected objects become visually highlighted
* The selected entity is available globally on the `Res<Selection>` resource
* Selected entities can also be queried via the `Selected` marker component
* Pressing `F` will "focus" the camera on the selected entity
* Pressing `Esc` will clear the active selection
### Known Issues
* We're currently using `bevy_mod_picking`'s "Selection" feature, which isn't
aware of our own input events. As a consequence, it will select the object
under the cursor when releasing `LMB` after orbiting the camera, which is not
what we want. This should be pretty easily fixable by dropping
`bevy_mod_picking`'s selection feature and reading the object under the
cursor in our own click listener that only runs if the LMB event is
unhandled.
* The skinned mesh is not currently selectable. This is likely fixable by
adjusting the logic in `scene::mark_scene_objects`... somehow. The `Gltf`
data structure is kind of a nightmare to work with, so this will take some
investigation.
### Future Enhancements
* For parity with other DCC apps, the "Focus" action should also adjust the
camera zoom level to "frame" the selected object in the view. Punting on that
for now because I'm not super confident about the math involved.
* Group selection with Shift/Ctrl and/or marquee selection would be really nice
to have, but probably not critical for now.
### New `math` module
* Traits for error-tolerant floating-point comparisons
* Traits for interpolation and easing
* Bevy plugin to manage 3D interpolations with ECS components
### `skybox` module refactor
* ECS resource to enable external skybox selection/updates
* System run conditions for more idiomatic/ECS-friendly control flow
### Camera controller refactor
* Renamed from `orbit_cam` to `camera_controller`
* Now delegates to the new interpolation plugin for WASD motion smoothing
* Simple API for attaching the `CameraController` component to the camera from
another module -- this resolves an awkward separation-of-concerns situation
where we had to configure a bunch of render-quality settings in the camera
controller module
### Scene module refactor
* Renamed from `test_scene` to `scene`
* This module is now responsible for configuring the skybox
* This module is now responsible for spawning the camera and configuring the
render settings
Brooks Palin