Bootstrap Image Generator application (#53)

This PR begins scaffolding the Stable Diffusion Turbo application (title
TBD), which is a separate frontend for Storyteller Engine which focuses
on image generation.

### Design Overview

* The application renders a "split-screen" view, with Storyteller Studio
  on the left, and the Stable Diffusion Turbo output on the right
* When changes are made in the Studio editor:
  - A screenshot is taken from the Bevy application
  - Screenshot is piped into Stable Diffusion Turbo model
  - Model output is rendered to the screen
* This all happens in as close to real-time as possible

### Work completed in this PR

* The `studio-frontend` project has been (somewhat crudely) refactored
  to split most of its components into reusable libraries
  - `studio-ui` is a use-case-agnostic design system library for core UI
    components. These are "dumb" components which are controlled
    via `@property` inputs and send `CustomEvent`s in response to user
    input.
  - `studio-frontend` is a collection of reusable Storyteller Studio
    widgets. These are "smart" components which depend on
    `@storyteller/studio` directly, manage their own state, and require very
    little configuration or styling. They are designed to be drop-in
    building blocks that can be reused between various frontends.
* The new application has been bootstrapped as `image-generator-demo`
  - Currently, this simply grabs screenshots from the Bevy application and
    renders those screenshots on the right side of the split-screen view. A
    future PR will wire up to the Storyteller.ai backend to pipe these
    screenshots through Stable Diffusion Turbo, and render that model's
    output instead.

### Testing

* Install dependencies as described in the repo's README
* Run `npx nx serve image-generator-demo`
* Open [localhost:4200](http://localhost:4200) in your browser
  - Use the camera controller in the left side of the view to manipulate
    the Bevy scene
  - Verify the right side of the view updates with a fresh screenshot at a
    rate of ~5 fps
* Toggle the "Realtime Capture" checkbox
  - Verify the right side view no longer updates while the user is
    actively manipulating the editor
  - This should allow the app to run smoother on lower-spec hardware

---------

Co-authored-by: Danny McGee <dannymcgee@gmail.com>
This commit is contained in:
Brooks Palin
2024-02-26 15:13:07 -07:00
committed by GitHub
parent 7b06d48c5d
commit d51eec52c3
71 changed files with 1331 additions and 579 deletions
+88
View File
@@ -0,0 +1,88 @@
import { on } from "@storyteller/framework";
import * as studio from "@storyteller/studio";
import { type EntitySelectEvent } from "@storyteller/studio";
import { LitElement, html, nothing, unsafeCSS } from "lit";
import { customElement, property, state } from "lit/decorators.js";
import { classMap } from "lit/directives/class-map.js";
import { ifDefined } from "lit/directives/if-defined.js";
import "./icon.element";
import styles from "./tree.element.scss?inline";
export interface Tree {
id?: string;
name: string;
children?: Tree[];
}
@customElement("sts-tree")
export class TreeElement extends LitElement {
static override styles = unsafeCSS(styles);
@property() name = "";
@property() icon?: string;
@property({ attribute: false }) treeChildren?: Tree[];
@property({ type: Boolean }) expanded = true;
@state() selected = false;
toggleExpanded(): void {
this.expanded = !this.expanded;
}
@on("window:entity-select")
onEntitySelect({ detail: entity }: EntitySelectEvent): void {
this.selected = this.id === entity;
}
selectEntity(): void {
if (this.id)
studio.select(this.id);
}
// TODO: A11y
protected override render = () => html`
<header
class=${classMap({
selected: this.selected,
selectable: !!this.id,
})}
>
${this.treeChildren?.length ? html`
<button class="toggle-expanded"
@click=${this.toggleExpanded}
>
<sts-icon
class=${classMap({
icon: true,
expanded: this.expanded,
})}
icon="caret-right"
aria-label=${this.expanded ? "Expand" : "Collapse"}
></sts-icon>
</button>
` : nothing}
<span class="label"
@click=${this.selectEntity}
>
${this.icon ? html`
<sts-icon icon=${this.icon}></sts-icon>
` : nothing}
${this.name}
</span>
</header>
${this.expanded && this.treeChildren?.length ? html`
<div class="children">
${this.treeChildren?.map(tree => html`
<sts-tree
id=${ifDefined(tree.id)}
.name=${tree.name}
.treeChildren=${tree.children}
></sts-tree>
`)}
</div>
` : nothing}
`;
}