mirror of
https://github.com/storytold/storyteller-bevy.git
synced 2026-10-18 00:23:14 +00:00
Transform gizmos (#10)
This PR adds functional transform gizmos, along with some new frontend UI to configure their state. ## Work Done - [x] Global / Local space toggling - [x] Gizmo type toggling - [x] Translation gizmo - [x] Axis-constrained translation - [x] Plane-constrained translation - [x] Screen-space translation - [x] Rotation gizmo - [x] Axis-constrained rotation - [x] Screen-space rotation - [x] "Trackball" rotation - [ ] Scale gizmo - [ ] Single-axis non-uniform scaling - [ ] Two-axis non-uniform scaling - [ ] Uniform scaling - [ ] Combination gizmos - [x] Translation + Rotation - [ ] Translation + Scale - [ ] Rotation + Scale - [ ] Translation + Rotation + Scale ## Future Enhancements ### Better visual feedback during transformations Most DCCs update the visual representation of the transform gizmo while it's in the process of being actively manipulated. (For example, Blender draws lines in world-space to indicate that the transformation is constrained to a particular axis or plane, hides widgets that aren't participating in the current transformation, and illustrates the angle of difference between the original and updated rotation.) At a minimum, we should indicate the current widget being manipulated with a color change (like we already do when the widget is being hovered). This requires a bit of additional state management effort but shouldn't be overly difficult to add in a future PR. ### Better visual representation for rotation widgets Currently, each axis of rotation is represented on the gizmo with a full-circle, unlit "ring" shape. This can make it difficult to tell which half of the ring is closer to the viewer, and thus to get an accurate understanding of how the transform is currently oriented, without moving the camera around. A [previous version of this work](#4) bisected the rings along a view-oriented plane and rendered the "back-side" ring halves with reduced opacity. This PR uses static meshes instead of an immediate-mode drawing API to render the gizmos, so a custom shader would be the ideal way to reproduce that effect. That effort should be deferred until after upgrading to Bevy 0.12, as it brings some changes which would likely require some updates to that code. ### Better visual representation for translation arrows A [previous version of this work](#4) faded the opacity of the translation arrows when they were very close to parallel with the view direction — an idea that was borrowed from Blender. It's a nice UX enhancement because translating along an axis that's very close to parallel with the view direction can be janky and error-prone, so it makes sense to disallow it and instead provide an unobstructed view of the screen-space translation widget. This would also require a custom shader, in addition to some extra state management logic (to disable the widgets when their opacity is zeroed out). Given that it's only a "nice-to-have," I think it makes sense to backlog this for the time being. ## Known Issues ### Picking There's currently a bug in `bevy_mod_picking` that affects the hit-testing of the gizmo widgets. The gizmos are currently _rendering_ on top of everything else (via a second camera and render-layer configurations), but the picking behavior doesn't take the rendering order into account. This means that portions of the gizmo widgets which are occluded by geometry in the world cannot be interacted with (because the occluding geometry interrupts the picking raycast). > Update: This is fixed in the new release of `bevy_mod_picking` for Bevy 0.12. ### Delay on first-render The first time a new widget shape is spawned, there's a noticeable delay (of 1 to several seconds) before the widget is rendered, during which the UI becomes unresponsive. This is likely the same issue that triggers a much longer delay when the glTF scene is being spawned. I'd like to see if Bevy 0.12 resolves or mitigates this issue before addressing it on our end, but if it doesn't, I think we can simply spawn an instance of each of the gizmo widgets off-screen on application startup to prevent lag spikes from happening while a user is interacting with the scene. ### Non-deterministic hover states The widget colors are currently being updated by, e.g., multiplying on mouser-over and dividing on mouse-out. Occasionally those events happen in quick enough succession that our mouse-out listener is not invoked, so a widget gets "stuck" in its hovered state. The fix for this is trivial but fairly annoying, so I haven't gotten around to it yet. 🙂 > Update: This issue mostly occurs when in world-space mode. The gizmo is torn > down and re-built from scratch once a drag event completes, which can despawn > the gizmo while an element is still hovered. The solution will likely involve > completely separate materials for default and hover states, similar to how > `bevy_mod_picking`'s highlight plugin works. ### Object Selection There's still some jank happening with object selection and camera orbiting, as mentioned in #8. I don't anticipate this being overly difficult to resolve, but I'm deferring that work until more of the core functionality is built out since it's not a blocker for now. ### Skinned Mesh Selection There's also still the issue of the skinned mesh being unselectable — which, it turns out, is [technically not actually the case](https://github.com/bevyengine/bevy/issues/4971). Rather, the picking raycast "sees" the mesh as if it were still in its original rest pose, because the mesh deformations are processed on the GPU and not reflected back to the CPU-side representation. I think the ideal workaround for this will be to attach capsule-shaped Rapier colliders to the skeleton (similar to Unreal Engine's auto-generated "physics assets" for skeletal meshes) and use those for picking instead. This would automatically enable skeletal pose manipulation (by transforming the individual joints), which is already a long term goal. It also makes more sense from a practical standpoint, as the mesh itself cannot be independently transformed while it's bound to the skeleton, so there's little point in directly selecting it.
This commit is contained in:
Danny McGee
@@ -10,6 +10,87 @@
|
||||
right: 16px;
|
||||
}
|
||||
|
||||
.toolbar {
|
||||
display: flex;
|
||||
gap: 16px;
|
||||
position: absolute;
|
||||
top: 16px;
|
||||
left: 50%;
|
||||
transform: translateX(-50%);
|
||||
}
|
||||
|
||||
.toolbar-group {
|
||||
display: flex;
|
||||
gap: 2px;
|
||||
}
|
||||
|
||||
.btn-toggle {
|
||||
display: inline-flex;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
position: relative;
|
||||
font-size: 16px;
|
||||
padding: 10px 12px;
|
||||
pointer-events: auto;
|
||||
|
||||
// TODO: Copypasta
|
||||
background: #0006;
|
||||
color: #FFFC;
|
||||
backdrop-filter: blur(16px);
|
||||
transition: {
|
||||
property: background, color;
|
||||
duration: 0.1s;
|
||||
timing-function: linear;
|
||||
}
|
||||
|
||||
&:first-child {
|
||||
border: {
|
||||
top-left-radius: 8px;
|
||||
bottom-left-radius: 8px;
|
||||
}
|
||||
}
|
||||
|
||||
&:last-child {
|
||||
border: {
|
||||
top-right-radius: 8px;
|
||||
bottom-right-radius: 8px;
|
||||
}
|
||||
}
|
||||
|
||||
&:hover {
|
||||
background: #0008;
|
||||
color: #FFF;
|
||||
}
|
||||
|
||||
&:has(:checked) {
|
||||
background: #7A2C34E3;
|
||||
color: #FFF;
|
||||
|
||||
&:hover {
|
||||
background: #90363F;
|
||||
}
|
||||
}
|
||||
|
||||
&:has(:disabled) {
|
||||
background: #0003 !important;
|
||||
color: #FFF6 !important;
|
||||
}
|
||||
|
||||
input {
|
||||
display: block;
|
||||
appearance: none;
|
||||
border: 0;
|
||||
margin: 0;
|
||||
padding: 0;
|
||||
opacity: 0;
|
||||
position: absolute;
|
||||
inset: 0;
|
||||
width: 100%;
|
||||
height: 100%;
|
||||
z-index: 1;
|
||||
}
|
||||
}
|
||||
|
||||
.help {
|
||||
position: absolute;
|
||||
top: 60px;
|
||||
|
||||
@@ -1,12 +1,13 @@
|
||||
import { on } from "@storyteller/framework";
|
||||
import * as studio from "@storyteller/studio";
|
||||
import { ViewportSize } from "@storyteller/studio";
|
||||
import { TransformSpace, TransformType, ViewportSize } from "@storyteller/studio";
|
||||
import { match } from "@storyteller/utility";
|
||||
import { LitElement, html, unsafeCSS } from "lit";
|
||||
import { customElement, state } from "lit/decorators.js";
|
||||
|
||||
import "./button.element";
|
||||
import "./help.element";
|
||||
import "./icons/fa-circle-info.element";
|
||||
import "./icon.element";
|
||||
|
||||
import styles from "./app.element.scss?inline";
|
||||
|
||||
@@ -16,6 +17,9 @@ export class AppElement extends LitElement {
|
||||
|
||||
@state() showHelp = false;
|
||||
|
||||
@state() transformSpace = TransformSpace.Local;
|
||||
@state() transformType = TransformType.Translate;
|
||||
|
||||
#studioCanvas?: HTMLCanvasElement | null;
|
||||
|
||||
override connectedCallback(): void {
|
||||
@@ -30,10 +34,71 @@ export class AppElement extends LitElement {
|
||||
studio.resize(new ViewportSize(width, height));
|
||||
}
|
||||
|
||||
@on("keydown")
|
||||
@on("keyup")
|
||||
pipeKeyboardEvents(event: KeyboardEvent): void {
|
||||
// When the user interacts with one of the web UI elements, the Bevy
|
||||
// canvas will lose focus, preventing it from receiving keyboard events.
|
||||
// As a workaround, we can listen for any events that have bubbled up to
|
||||
// the app root and repeat them for the canvas.
|
||||
//
|
||||
// This does unfortunately burden the web front-end with some additional
|
||||
// responsibility: when we handle a keyboard event and _don't_ want the
|
||||
// Bevy app to receive it, we'll need to call `event.stopPropagation()` in
|
||||
// the handler to prevent it from reaching this listener.
|
||||
const {
|
||||
type,
|
||||
code, charCode, key, keyCode, location, repeat,
|
||||
altKey, ctrlKey, metaKey, shiftKey,
|
||||
} = event;
|
||||
|
||||
this.#studioCanvas?.dispatchEvent(new KeyboardEvent(type, {
|
||||
code, charCode, key, keyCode, location, repeat,
|
||||
altKey, ctrlKey, metaKey, shiftKey,
|
||||
bubbles: true,
|
||||
cancelable: true,
|
||||
composed: true,
|
||||
}));
|
||||
}
|
||||
|
||||
toggleHelp(): void {
|
||||
this.showHelp = !this.showHelp;
|
||||
}
|
||||
|
||||
onTransformSpaceChange(event: InputEvent): void {
|
||||
const { value } = event.target as HTMLInputElement;
|
||||
const space = match (value, {
|
||||
"local": () => TransformSpace.Local,
|
||||
"world": () => TransformSpace.World,
|
||||
_: () => {
|
||||
throw new Error(`Unknown transform space: "${value}"`);
|
||||
}
|
||||
});
|
||||
|
||||
this.transformSpace = space;
|
||||
studio.setTransformSpace(space);
|
||||
}
|
||||
|
||||
onTransformTypeChange(event: InputEvent): void {
|
||||
const { name, checked } = event.target as HTMLInputElement;
|
||||
const flag = match (name, {
|
||||
"translate": () => TransformType.Translate,
|
||||
"rotate": () => TransformType.Rotate,
|
||||
"scale": () => TransformType.Scale,
|
||||
_: () => {
|
||||
throw new Error(`Unknown transform type: "${name}"`);
|
||||
}
|
||||
});
|
||||
|
||||
if (checked) {
|
||||
this.transformType |= flag;
|
||||
} else {
|
||||
this.transformType &= ~flag;
|
||||
}
|
||||
|
||||
studio.setTransformType(this.transformType);
|
||||
}
|
||||
|
||||
#initStudio(): HTMLCanvasElement | null {
|
||||
const width = window.innerWidth;
|
||||
const height = window.innerHeight;
|
||||
@@ -48,6 +113,58 @@ export class AppElement extends LitElement {
|
||||
}
|
||||
|
||||
protected override render = () => html`
|
||||
<!-- TODO: What's a "component"? -->
|
||||
<div class="toolbar" role="toolbar">
|
||||
<div class="toolbar-group" role="group">
|
||||
<label class="btn-toggle">
|
||||
<sts-icon icon="cube">Local</sts-icon>
|
||||
<input type="radio"
|
||||
name="transform-space"
|
||||
value="local"
|
||||
?checked=${this.transformSpace === TransformSpace.Local}
|
||||
@input=${this.onTransformSpaceChange}
|
||||
/>
|
||||
</label>
|
||||
<label class="btn-toggle">
|
||||
<sts-icon icon="globe">Global</sts-icon>
|
||||
<input type="radio"
|
||||
name="transform-space"
|
||||
value="world"
|
||||
?checked=${this.transformSpace === TransformSpace.World}
|
||||
@input=${this.onTransformSpaceChange}
|
||||
/>
|
||||
</label>
|
||||
</div>
|
||||
<div class="toolbar-group" role="group">
|
||||
<label class="btn-toggle">
|
||||
<sts-icon icon="up-down-left-right">Translate</sts-icon>
|
||||
<input type="checkbox"
|
||||
name="translate"
|
||||
?checked=${this.transformType & TransformType.Translate}
|
||||
@input=${this.onTransformTypeChange}
|
||||
/>
|
||||
</label>
|
||||
<!-- disabled -->
|
||||
<label class="btn-toggle">
|
||||
<sts-icon icon="rotate">Rotate</sts-icon>
|
||||
<input type="checkbox"
|
||||
name="rotate"
|
||||
?checked=${this.transformType & TransformType.Rotate}
|
||||
@input=${this.onTransformTypeChange}
|
||||
/>
|
||||
</label>
|
||||
<label class="btn-toggle">
|
||||
<sts-icon icon="scale">Scale</sts-icon>
|
||||
<input type="checkbox"
|
||||
name="scale"
|
||||
disabled
|
||||
?checked=${this.transformType & TransformType.Scale}
|
||||
@input=${this.onTransformTypeChange}
|
||||
/>
|
||||
</label>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<sts-button class="btn-help"
|
||||
icon="circle-info"
|
||||
@click=${this.toggleHelp}
|
||||
|
||||
@@ -1,10 +1,11 @@
|
||||
import { match } from "@storyteller/utility";
|
||||
import { on } from "@storyteller/framework";
|
||||
import { LitElement, html, unsafeCSS } from "lit";
|
||||
import { LitElement, html, nothing, unsafeCSS } from "lit";
|
||||
import { customElement, property } from "lit/decorators.js";
|
||||
|
||||
import styles from "./button.element.scss?inline";
|
||||
|
||||
import "./icon.element";
|
||||
|
||||
@customElement("sts-button")
|
||||
export class ButtonElement extends LitElement {
|
||||
static override styles = unsafeCSS(styles);
|
||||
@@ -15,16 +16,17 @@ export class ButtonElement extends LitElement {
|
||||
override tabIndex = 0;
|
||||
|
||||
@on("keydown")
|
||||
onKeyDown({ key }: KeyboardEvent) {
|
||||
if (/^( |Enter)$/.test(key))
|
||||
onKeyDown(event: KeyboardEvent) {
|
||||
if (/^( |Enter)$/.test(event.key)) {
|
||||
event.stopPropagation();
|
||||
this.click();
|
||||
}
|
||||
}
|
||||
|
||||
protected override render = () => html`
|
||||
${match (this.icon, {
|
||||
"circle-info": () => html`<fa-circle-info class="icon"></fa-circle-info>`,
|
||||
_: () => null,
|
||||
})}
|
||||
${this.icon
|
||||
? html`<sts-icon icon=${this.icon} aria-hidden="true"></sts-icon>`
|
||||
: nothing}
|
||||
<slot></slot>
|
||||
`
|
||||
}
|
||||
|
||||
@@ -0,0 +1,28 @@
|
||||
:host {
|
||||
display: flex;
|
||||
width: auto;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
height: 1em;
|
||||
}
|
||||
|
||||
.a11y-label {
|
||||
position: absolute;
|
||||
overflow: hidden;
|
||||
width: 1px;
|
||||
height: 1px;
|
||||
margin: -1px;
|
||||
padding: 0;
|
||||
clip: rect(1px, 1px, 1px, 1px);
|
||||
border: 0;
|
||||
white-space: nowrap;
|
||||
}
|
||||
|
||||
.svg-wrapper {
|
||||
display: flex;
|
||||
}
|
||||
|
||||
svg {
|
||||
height: 1em;
|
||||
fill: currentColor;
|
||||
}
|
||||
@@ -0,0 +1,65 @@
|
||||
import { spread } from "@storyteller/framework";
|
||||
import { match } from "@storyteller/utility";
|
||||
import { LitElement, TemplateResult, html, nothing, unsafeCSS } from "lit";
|
||||
import { customElement, property } from "lit/decorators.js";
|
||||
import { unsafeSVG } from "lit/directives/unsafe-svg.js";
|
||||
|
||||
import fasCircleInfo from "@fortawesome/fontawesome-pro/svgs/regular/circle-info.svg?raw";
|
||||
import fasCube from "@fortawesome/fontawesome-pro/svgs/regular/cube.svg?raw";
|
||||
import fasGlobe from "@fortawesome/fontawesome-pro/svgs/regular/globe.svg?raw";
|
||||
import fasUpDownLeftRight from "@fortawesome/fontawesome-pro/svgs/regular/arrows-up-down-left-right.svg?raw";
|
||||
import fasRotate from "@fortawesome/fontawesome-pro/svgs/regular/arrows-rotate.svg?raw";
|
||||
import fasScale from "@fortawesome/fontawesome-pro/svgs/regular/arrow-up-right-and-arrow-down-left-from-center.svg?raw";
|
||||
|
||||
import styles from "./icon.element.scss?inline";
|
||||
|
||||
@customElement("sts-icon")
|
||||
export class IconElement extends LitElement {
|
||||
static override styles = unsafeCSS(styles);
|
||||
|
||||
@property() icon = "";
|
||||
|
||||
protected override render = () => html`
|
||||
<span class="a11y-label">
|
||||
<slot></slot>
|
||||
</span>
|
||||
<div class="svg-wrapper" role="presentation">
|
||||
${renderSvg(this.icon)}
|
||||
</div>
|
||||
`;
|
||||
}
|
||||
|
||||
const RENDER_RESULTS = new Map<string, TemplateResult>();
|
||||
|
||||
function renderSvg(iconId: string) {
|
||||
if (RENDER_RESULTS.has(iconId))
|
||||
return RENDER_RESULTS.get(iconId)!;
|
||||
|
||||
const svgSrc = match (iconId, {
|
||||
"circle-info": () => fasCircleInfo,
|
||||
"cube": () => fasCube,
|
||||
"globe": () => fasGlobe,
|
||||
"up-down-left-right": () => fasUpDownLeftRight,
|
||||
"rotate": () => fasRotate,
|
||||
"scale": () => fasScale,
|
||||
_: () => null,
|
||||
});
|
||||
|
||||
if (!svgSrc) return nothing;
|
||||
|
||||
const [, attrsString, body] = svgSrc.match(/<svg(.+?)>(.+?)<\/svg>/)!;
|
||||
const attrs = Object.fromEntries(
|
||||
Array.from(attrsString.matchAll(/([-:_a-zA-Z0-9]+)(?:=['"]?([^'"]*)['"]?)/g))
|
||||
.map(([, attr, value]) => [attr, value])
|
||||
);
|
||||
|
||||
const result = html`
|
||||
<svg ${spread(attrs)}>
|
||||
${unsafeSVG(body)}
|
||||
</svg>
|
||||
`;
|
||||
|
||||
RENDER_RESULTS.set(iconId, result);
|
||||
|
||||
return result;
|
||||
}
|
||||
@@ -1,20 +0,0 @@
|
||||
import { LitElement, html } from "lit";
|
||||
import { customElement } from "lit/decorators.js";
|
||||
|
||||
@customElement("fa-circle-info")
|
||||
export class CircleInfoIcon extends LitElement {
|
||||
override render = () => html`
|
||||
<svg xmlns="http://www.w3.org/2000/svg"
|
||||
height="1em"
|
||||
viewBox="0 0 512 512"
|
||||
>
|
||||
<!--!
|
||||
Font Awesome Free 6.4.2 by @fontawesome - https://fontawesome.com
|
||||
License - https://fontawesome.com/license (Commercial License)
|
||||
Copyright 2023 Fonticons, Inc.
|
||||
-->
|
||||
<path d="M256 512A256 256 0 1 0 256 0a256 256 0 1 0 0 512zM216 336h24V272H216c-13.3 0-24-10.7-24-24s10.7-24 24-24h48c13.3 0 24 10.7 24 24v88h8c13.3 0 24 10.7 24 24s-10.7 24-24 24H216c-13.3 0-24-10.7-24-24s10.7-24 24-24zm40-208a32 32 0 1 1 0 64 32 32 0 1 1 0-64z"
|
||||
/>
|
||||
</svg>
|
||||
`;
|
||||
}
|
||||
Reference in New Issue
Block a user