This PR adds an initial animation timeline.
---------
Co-authored-by: Brandon Thomas <echelon@gmail.com>
Co-authored-by: Danny McGee <dannymcgee@gmail.com>
This PR begins scaffolding the Stable Diffusion Turbo application (title
TBD), which is a separate frontend for Storyteller Engine which focuses
on image generation.
### Design Overview
* The application renders a "split-screen" view, with Storyteller Studio
on the left, and the Stable Diffusion Turbo output on the right
* When changes are made in the Studio editor:
- A screenshot is taken from the Bevy application
- Screenshot is piped into Stable Diffusion Turbo model
- Model output is rendered to the screen
* This all happens in as close to real-time as possible
### Work completed in this PR
* The `studio-frontend` project has been (somewhat crudely) refactored
to split most of its components into reusable libraries
- `studio-ui` is a use-case-agnostic design system library for core UI
components. These are "dumb" components which are controlled
via `@property` inputs and send `CustomEvent`s in response to user
input.
- `studio-frontend` is a collection of reusable Storyteller Studio
widgets. These are "smart" components which depend on
`@storyteller/studio` directly, manage their own state, and require very
little configuration or styling. They are designed to be drop-in
building blocks that can be reused between various frontends.
* The new application has been bootstrapped as `image-generator-demo`
- Currently, this simply grabs screenshots from the Bevy application and
renders those screenshots on the right side of the split-screen view. A
future PR will wire up to the Storyteller.ai backend to pipe these
screenshots through Stable Diffusion Turbo, and render that model's
output instead.
### Testing
* Install dependencies as described in the repo's README
* Run `npx nx serve image-generator-demo`
* Open [localhost:4200](http://localhost:4200) in your browser
- Use the camera controller in the left side of the view to manipulate
the Bevy scene
- Verify the right side of the view updates with a fresh screenshot at a
rate of ~5 fps
* Toggle the "Realtime Capture" checkbox
- Verify the right side view no longer updates while the user is
actively manipulating the editor
- This should allow the app to run smoother on lower-spec hardware
---------
Co-authored-by: Danny McGee <dannymcgee@gmail.com>
* Migrate `System::pipe(...)` calls to `System::map(...)`
* Migrate `EventReader::iter()` and `EventReader::into_iter()` calls to
`EventReader::read()`
* Migrate asset handling
* Generate `*.meta` files for assets
* Migrate a shader import
* Fix emissive materials double-applying their emissive strength
* Fix non-normalized bone weights resulting in incorrect vertex positions
* Set shadow filtering to `Hardware2x2` to mitigate increase in shader
compilation times
This PR adds the bones of a user-facing scene inspector.
### Work Done
To facilitate this work, the Bevy/frontend interface has been expanded to
accommodate two-way communications:
- The Bevy app can now dispatch JavaScript `CustomEvent`s via
`wasm::dispatch_to_js(...)`
- The frontend application can receive those events by listening for the given
event type on the `window` object
In its current state, the inspection system works by attaching the `Inspectable`
marker component to any entities which should be exposed to the user. A Bevy
system runs on `Update` to check for newly spawned or despawned entities with
that component, and emits events to notify the frontend. The frontend listens
for those events and updates its model of the scene accordingly, which is then
displayed with a simple tree view.
### Future Enhancements
There are many improvements that must be made to make this system actually
useful, but this PR is being expedited to focus on migrating to Bevy 0.12. A
shortlist of planned updates:
- Identify and expose the "type" of entity (e.g. lights, cameras, meshes,
skeletons, etc.) via a field on the `Inspectable` component
- Highlight the currently selected entity in the frontend hierarchy
- Ask the Bevy app to update the selection when the user selects an entry in the
hierarchy
- Add a separate UI panel to inspect and modify the selected entity's components
via the frontend
rewin