Fix some issues causing input jank (#106)

This PR fixes two issues that can cause a degraded user experience under
certain conditions:

### Focus-stealing

When focus lands on the `document.body`, it's typically bad for
accessibility and a sign of improper input handling. This generally
occurs when the currently-focused element is removed from the DOM. In
our particular case, it also prevents the Bevy application from
correctly handling keyboard events.

Previously, the problem of `document.body` focus had been fixed by
checking the `document.activeElement` every frame, and redirecting focus
to the Bevy canvas whenever it landed on the `document.body`. However,
when running in an iframe embedded on https://fakeyou.com, our
application will report `document.body` as the `document.activeElement`
whenever the parent application has focus, effectively stealing focus
from the parent application and making it impossible for the user to,
e.g., interact with text fields in the parent app.

This PR removes that focus-stealing behavior. Instead, we should
properly implement the ARIA specifications for any elements that may
disappear from the DOM so that user focus is managed in a precise and
predictable way. That will be addressed in follow-up PRs.

### "Stuck" input states

In certain circumstances, the browser window hosting our application may
lose focus while certain keys/buttons are held down which are bound to
long-running actions in the Bevy application, like camera manipulation.
When that occurs, Bevy is unable to detect when the bound keys/buttons
are released, causing the input state to become "stuck" reading those
keys/buttons as being pressed.

As a workaround, this PR introduces a listener for "blur" events on the
`window`, which dispatches synthetic "keyup"/"pointerup" events to the
Bevy canvas for all affected keyboard keys and mouse buttons.
This commit is contained in:
Danny McGee
2024-03-26 19:36:47 -04:00
committed by GitHub
parent b9ec09e06c
commit 21a6cb42fe
4 changed files with 284 additions and 26 deletions
+67 -6
View File
@@ -39,27 +39,88 @@ impl Plugin for StudioInputPlugin {
.after(bevy::input::InputSystem)
.after(read_mouse_delta),
);
app.add_systems(
PostUpdate,
(apply_deferred, log_input_actions).after(StudioInputSystem),
);
}
}
#[derive(Event, Clone, Copy)]
/// Logs all `InputAction`s emitted, along with the input state at the time they
/// were fired, with `debug` verbosity.
fn log_input_actions(
ri_keyboard: Res<Input<KeyCode>>,
ri_mouse_buttons: Res<Input<MouseButton>>,
mut er_input_actions: EventReader<InputAction>,
) {
if er_input_actions.is_empty() {
return;
}
let lmb = if ri_mouse_buttons.pressed(MouseButton::Left) {
'X'
} else {
'-'
};
let mmb = if ri_mouse_buttons.pressed(MouseButton::Middle) {
'X'
} else {
'-'
};
let rmb = if ri_mouse_buttons.pressed(MouseButton::Right) {
'X'
} else {
'-'
};
use KeyCode::*;
let mod_keys = [AltLeft, AltRight, ShiftLeft, ControlLeft, ControlRight]
.into_iter()
.collect::<HashSet<_>>();
let action_keys = [W, A, S, D, E, Q, Escape, Grave, Delete, F]
.into_iter()
.collect::<HashSet<_>>();
let keys_pressed = ri_keyboard.get_pressed().copied().collect::<HashSet<_>>();
let keys_display = keys_pressed
.intersection(&mod_keys)
.map(|key| format!("{key:?}"))
.chain(
keys_pressed
.intersection(&action_keys)
.map(|key| format!("{key:?}")),
)
.collect::<Vec<_>>()
.join("+");
for action in er_input_actions.read() {
debug!(
"Mouse: [{lmb}{mmb}{rmb}] \
Keyboard: [{keys_display}] \
Action: [{action:.2?}]"
);
}
}
#[derive(Event, Clone, Copy, Debug)]
pub struct InputAction(pub Lifecycle, pub Action);
#[derive(Clone, Copy)]
#[derive(Clone, Copy, Debug)]
pub enum Lifecycle {
Start,
Ongoing,
Stop,
}
#[derive(Clone, Copy)]
#[derive(Clone, Copy, Debug)]
pub enum Action {
Camera(CameraMotion),
Selection(Selection),
ToggleDebug,
}
#[derive(Clone, Copy)]
#[derive(Clone, Copy, Debug)]
pub enum CameraMotion {
Translate(Vec3),
Pan(Vec2),
@@ -73,7 +134,7 @@ impl From<CameraMotion> for Action {
}
}
#[derive(Clone, Copy)]
#[derive(Clone, Copy, Debug)]
pub enum Selection {
/// Clear the current selection
Clear,
@@ -206,7 +267,7 @@ fn translate_dcc_camera_events(
CameraMotion::Rotate(Vec2::ZERO).into(),
));
}
// Alt Camera Rotation (designed for laptop)
// Alternative Camera Rotation (designed for laptop)
else if keyboard.pressed(KeyCode::ShiftLeft) {
let lifecycle = if keyboard.just_pressed(KeyCode::ShiftLeft) {
Lifecycle::Start