storytold/storyteller-bevy
2
0
Fork 0
mirror of https://github.com/storytold/storyteller-bevy.git synced 2026-10-26 08:38:10 +00:00
Code Issues Packages Projects Releases Wiki Activity

Bootstrap Image Generator application (#53)

Browse Source 
This PR begins scaffolding the Stable Diffusion Turbo application (title
TBD), which is a separate frontend for Storyteller Engine which focuses
on image generation.

### Design Overview

* The application renders a "split-screen" view, with Storyteller Studio
  on the left, and the Stable Diffusion Turbo output on the right
* When changes are made in the Studio editor:
  - A screenshot is taken from the Bevy application
  - Screenshot is piped into Stable Diffusion Turbo model
  - Model output is rendered to the screen
* This all happens in as close to real-time as possible

### Work completed in this PR

* The `studio-frontend` project has been (somewhat crudely) refactored
  to split most of its components into reusable libraries
  - `studio-ui` is a use-case-agnostic design system library for core UI
    components. These are "dumb" components which are controlled
    via `@property` inputs and send `CustomEvent`s in response to user
    input.
  - `studio-frontend` is a collection of reusable Storyteller Studio
    widgets. These are "smart" components which depend on
    `@storyteller/studio` directly, manage their own state, and require very
    little configuration or styling. They are designed to be drop-in
    building blocks that can be reused between various frontends.
* The new application has been bootstrapped as `image-generator-demo`
  - Currently, this simply grabs screenshots from the Bevy application and
    renders those screenshots on the right side of the split-screen view. A
    future PR will wire up to the Storyteller.ai backend to pipe these
    screenshots through Stable Diffusion Turbo, and render that model's
    output instead.

### Testing

* Install dependencies as described in the repo's README
* Run `npx nx serve image-generator-demo`
* Open [localhost:4200](http://localhost:4200) in your browser
  - Use the camera controller in the left side of the view to manipulate
    the Bevy scene
  - Verify the right side of the view updates with a fresh screenshot at a
    rate of ~5 fps
* Toggle the "Realtime Capture" checkbox
  - Verify the right side view no longer updates while the user is
    actively manipulating the editor
  - This should allow the app to run smoother on lower-spec hardware

---------

Co-authored-by: Danny McGee <dannymcgee@gmail.com>
This commit is contained in:
Brooks Palin
2024-02-26 15:13:07 -07:00
committed by GitHub
parent 7b06d48c5d
commit d51eec52c3
71 changed files with 1331 additions and 579 deletions
Download Patch File Download Diff File Expand all files Collapse all files
studio-web/src/lib/transform-toolbar.element.scss
+57
View File
@@ -0,0 +1,57 @@
.btn-toggle {
display: inline-flex;
align-items: center;
justify-content: center;
position: relative;
font-size: 16px;
padding: 10px 12px;
pointer-events: auto;
// TODO: Copypasta
background: #0006;
color: #FFFC;
backdrop-filter: blur(16px);
transition: {
property: background, color;
duration: 0.1s;
timing-function: linear;
}
&:hover {
background: #0008;
color: #FFF;
}
&:has(:focus-visible) {
z-index: 1;
outline: 4px solid #2F83FFBF;
}
&:has(:checked) {
background: #7A2C34E3;
color: #FFF;
&:hover {
background: #90363F;
}
}
&:has(:disabled) {
background: #0003 !important;
color: #FFF6 !important;
}
input {
display: block;
appearance: none;
border: 0;
margin: 0;
padding: 0;
opacity: 0;
position: absolute;
inset: 0;
width: 100%;
height: 100%;
z-index: 1;
}
}