mirror of
https://github.com/storytold/storyteller-ml.git
synced 2026-10-16 00:13:07 +00:00
Justin John
150 lines
6.8 KiB
Markdown
150 lines
6.8 KiB
Markdown
# MocapNET V4 Documentation
|
|
|
|
Welcome to the MocapNET V4 documentation! 🎉 Immerse yourself in the world of advanced motion capture technology with ease.
|
|
|
|
🔗 **Find the Original Documentation** at [README_ORIGINAL.md](./README_ORIGINAL.md).
|
|
|
|
This version from the `tag mnet4` branch is sourced directly from the [official MocapNET repository](https://github.com/FORTH-ModelBasedTracker/MocapNET/tree/mnet4).
|
|
|
|
## 🚀 Getting Started: Developer Installation via docker (For Server Deployment Usage)
|
|
|
|
To install MocapNET V4 on a Linux system:
|
|
|
|
1. **Clone the MocapNET Repository**:
|
|
```sh
|
|
git clone https://github.com/storytold/storyteller-ml/master/animation/MocapNET.git
|
|
```
|
|
|
|
2. **Deployment**:
|
|
```sh
|
|
cd MocapNET/docker
|
|
bash build_and_deploy.sh
|
|
```
|
|
|
|
This script will build and run a Docker image, setting up the necessary environment for MocapNET V4.
|
|
|
|
______________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________
|
|
|
|
### 🚀 Getting Started: Main Installation Process (Run locally)
|
|
|
|
For both Linux and Windows (via WSL), follow these steps:
|
|
|
|
1. **Install Dependencies**:
|
|
```sh
|
|
sudo apt-get install git build-essential cmake libopencv-dev libjpeg-dev libpng-dev libglew-dev libpthread-stubs0-dev
|
|
```
|
|
|
|
2. **Clone the MocapNET Repository**:
|
|
```sh
|
|
git clone https://github.com/storytold/storyteller-ml/master/animation/MocapNET.git
|
|
```
|
|
|
|
3. **Navigate to MocapNET Directory**:
|
|
```sh
|
|
cd MocapNET
|
|
```
|
|
|
|
4. **Run the Initialization Script**:
|
|
```sh
|
|
./initialize.sh
|
|
```
|
|
|
|
This will set up MocapNET V4 on your system, including all necessary dependencies.
|
|
|
|
______________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________
|
|
|
|
### Additional Steps for Windows Users
|
|
|
|
For Windows users, using the Windows Subsystem for Linux (WSL) is recommended:
|
|
|
|
1. **Enable WSL on Windows**:
|
|
- Open PowerShell as Administrator and execute:
|
|
```powershell
|
|
dism.exe /online /enable-feature /featurename:Microsoft-Windows-Subsystem-Linux /all /norestart
|
|
```
|
|
- Restart your computer when prompted.
|
|
|
|
2. **Install a Linux Distribution from the Microsoft Store**:
|
|
- Choose and install your preferred Linux distribution (e.g., Ubuntu, Debian, Fedora).
|
|
|
|
3. **Set Up Your Linux Distribution**:
|
|
- Launch the Linux distribution and complete the account setup.
|
|
|
|
4. **Update and Upgrade Your Linux Distribution**:
|
|
- Run `sudo apt update && sudo apt upgrade` in the Linux terminal.
|
|
|
|
For a detailed WSL setup guide, view [How to Set Up WSL](https://www.youtube.com/watch?v=1HzYU2_t3yc) on YouTube.
|
|
|
|
______________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________
|
|
|
|
## 🎬 How to Use MocapNET V4
|
|
|
|
### Preparing for Usage
|
|
|
|
Before running MocapNET V4, navigate to the appropriate directory:
|
|
|
|
```sh
|
|
cd src/python/mnet4
|
|
```
|
|
|
|
### Quick Start with a Sample Video
|
|
|
|
Now you're ready to download and use a sample video:
|
|
|
|
1. **Download a Sample Video**:
|
|
```sh
|
|
wget http://ammar.gr/mocapnet/shuffle.webm -O shuffle.webm
|
|
```
|
|
|
|
2. **Process the Video**:
|
|
Execute the following command to run MocapNET on the downloaded video:
|
|
```sh
|
|
python3 -m mediapipeHolisticWebcamMocapNET --from shuffle.webm --ik 0.001 99 99 --smooth 60 10 --all --save --plot --headless 2
|
|
```
|
|
|
|
### Convert Output BVH
|
|
|
|
For further processing, such as generating a BVH file without face and toes bones, execute:
|
|
|
|
```sh
|
|
./GroundTruthDumper --from dependencies/RGBDAcquisition/opengl_acquisition_shared_library/opengl_depth_and_color_renderer/Motions/DAZFriendlyCGSPEED_ZXYAndHandsAxisBigHands.bvh --merge src/python/mnet4/out.bvh dependencies/RGBDAcquisition/opengl_acquisition_shared_library/opengl_depth_and_color_renderer/Motions//mergeDazFriendlyAndAddHead.profile --setPositionRotation 0 0 0 0 0 0 --bvh remade.bvh
|
|
```
|
|
______________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________
|
|
|
|
### Notes:
|
|
|
|
- The output bvh file will be stored inside this directory ``MocapNET/src/python/mnet4`` with the filename ``out.bvh``
|
|
|
|
- The converted bvh file will be stored in the root directory ``MocapNET`` with the filename ``remade.bvh``
|
|
|
|
______________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________
|
|
|
|
### Command-Line Options Explained
|
|
|
|
- `--headless`: Skip the live video display. Ideal for headless servers or GUI-less environments.
|
|
- `--live`: Launches a live demo. Functionalities may vary.
|
|
- `--mt`: Enable multi-threading for better performance on multi-core CPUs.
|
|
- `--calib [filename]`: Use a specific calibration file for camera adjustments.
|
|
- `--frameskip [number]`: Skip a set number of frames to reduce load or speed up processing.
|
|
- `--nnsubsample [number]`: Run the neural network at set intervals (e.g., every 2nd frame).
|
|
- `--ik [float int int]`: Set parameters for inverse kinematics optimization.
|
|
- `--smooth [float float]`: Apply smoothing to motion capture data.
|
|
- `--noik`: Turn off inverse kinematics processing.
|
|
- `--aspectCorrection [float]`: Adjust video or image aspect ratio.
|
|
- `--noise [float]`: Add noise to input for testing under varied conditions.
|
|
- `--size [int int]`: Define video/image dimensions.
|
|
- `--scale [float]`: Scale the output motion data.
|
|
- `--plot`: Plot BVH channels for skeletal animation visualization.
|
|
- `--all`: Activate all processing features (body, eyes, mouth, hands).
|
|
- `--nobody`: Disable body joint estimation.
|
|
- `--face`: Enable face joint estimation.
|
|
- `--eyes / --reye`: Estimate right eye joints.
|
|
- `--mouth`: Enable mouth joint estimation.
|
|
- `--hands`: Estimate hand joints.
|
|
- `--save`: Save outputs (images, videos, data, logs).
|
|
- `--engine [engine_name]`: Choose a specific neural network backend.
|
|
- `--from [filepath]`: Specify input source (video file or live feed).
|
|
- `--profile`: Activate performance profiling.
|
|
|
|
## THIS README IS A W.I.P
|