add scaling, docs, and adjust example_mapping

This commit is contained in:
Lee-Orr
2024-02-12 16:13:15 -05:00
parent dac13a18c1
commit d7eeaf4b70
3 changed files with 83 additions and 5 deletions
+73 -1
View File
@@ -1,3 +1,75 @@
# Bevy Facial Mocap
This is a project build for storyteller.ai, to provide support for importing ARKit data into bevy, and using it to drive shape keys.
This is a project build for storyteller.ai, to provide support for importing ARKit data into bevy, and using it to drive shape keys.
## Use
Import the asset loader:
```rust
app
.init_asset_loader::<CSVAnimationLoader>()
```
and in code, you can load assets just like any normal animation clip:
```rust
let handle = asset_server.load("recorded_test.morph_anim.json");
let mut player = AnimationPlayer::default();
player.play(handle.clone()).repeat();
```
## Asset Preparation
To load a recording, you need 3 files:
- a `.csv` with the ARKit data
- a `.json` file with the mapping info
- a `.morph_anim.json` file that points to the `.csv`, the mapping file, and provides the name of the animation root. Ideally, it would also provide an FPS - though that can be computed if it is an integer value (so not for 60 or 30, but not for 29.97)
### .morph_anim.json structure
This is the structure of the animation file itself:
```json
{
"csv": "./the_relative/path_to_the.csv",
"root_name": "pascal_Head_or_whatever_root_entity_you_want_here",
"fps": 4,
"mapping": "the_relative/path_to_the/mapping.json"
}
```
### Mapping .json structure
The mapping .json is a json list/array. Each item is a sub-list where the first element is the mapping, and the second element is the name of the morph in the CSV file.
For example:
```json
[
["eyeBlinkLeft", "EyeBlinkLeft"],
["eyeLookDownLeft", "EyeLookDownLeft"]
]
```
For morphs, the sub lists need to be in the same order as the morphs in the GLTF files.
For bones, the first element of the sub-list is the mapping information. It provides the path to the bone (from root), the axis of the transform, and an optional offset or scale:
```json
[{
"path": ["pascal_Head"],
"axis": "Y",
"scale": 0.4
}, "HeadYaw"],
[{
"path": ["pascal_Head"],
"axis": "X",
"offset": -45
}, "HeadPitch"],
[{
"path": ["pascal_Head"],
"axis": [0, 0, 1]
}, "HeadRoll"]
```
The [example_mapping.json](assets/example_mapping.json) file contains a full example of the mapping. And the [simple example](examples/simple_example.rs) contains a full code sample you can try by running `cargo run --example simple_example`.
+6 -3
View File
@@ -53,15 +53,18 @@
["tongueOut", "TongueOut"],
[{
"path": ["pascal_Head"],
"axis": "Y"
"axis": "Y",
"scale": 0.4
}, "HeadYaw"],
[{
"path": ["pascal_Head"],
"axis": "X",
"offset": -45
"offset": -22,
"scale": 0.4
}, "HeadPitch"],
[{
"path": ["pascal_Head"],
"axis": "Z"
"axis": "Z",
"scale": 0.4
}, "HeadRoll"]
]
+4 -1
View File
@@ -293,7 +293,9 @@ impl CSVAnimation {
};
let offset = mapping.offset.unwrap_or(0f32).to_radians();
let result = result * 180f32.to_radians() + offset;
let scale = mapping.scale.unwrap_or(1f32);
let result = result * 180f32.to_radians() * scale + offset;
println!("Rotating {axis:?} by {result:?}");
println!("From {}", transform.rotation.xyz());
@@ -386,6 +388,7 @@ pub struct BoneMapping {
path: Vec<String>,
axis: Axis,
offset: Option<f32>,
scale: Option<f32>
}
#[derive(Debug, Clone, Serialize, Deserialize)]