storyteller-ml/animation/MocapNET

MocapNET V4 Documentation

Welcome to the MocapNET V4 documentation! 🎉 Immerse yourself in the world of advanced motion capture technology with ease.

🔗 Find the Original Documentation at README_ORIGINAL.md.

This version from the tag mnet4 branch is sourced directly from the official MocapNET repository.

🚀 Getting Started: Developer Installation via docker (For Server Deployment Usage)

To install MocapNET V4 on a Linux system:

1. Clone the MocapNET Repository:

   git clone https://github.com/storytold/storyteller-ml/master/animation/MocapNET.git
   

2. Deployment:

cd MocapNET/docker
bash build_and_deploy.sh

This script will build and run a Docker image, setting up the necessary environment for MocapNET V4.

---

🚀 Getting Started: Main Installation Process (Run locally)

For both Linux and Windows (via WSL), follow these steps:

1. Install Dependencies:

   sudo apt-get install git build-essential cmake libopencv-dev libjpeg-dev libpng-dev libglew-dev libpthread-stubs0-dev
   

2. Clone the MocapNET Repository:

   git clone https://github.com/storytold/storyteller-ml/master/animation/MocapNET.git
   

3. Navigate to MocapNET Directory:

   cd MocapNET
   

4. Run the Initialization Script:

   ./initialize.sh
   

This will set up MocapNET V4 on your system, including all necessary dependencies.

---

Additional Steps for Windows Users

For Windows users, using the Windows Subsystem for Linux (WSL) is recommended:

1. Enable WSL on Windows:

   • Open PowerShell as Administrator and execute:

     dism.exe /online /enable-feature /featurename:Microsoft-Windows-Subsystem-Linux /all /norestart
     

   • Restart your computer when prompted.

2. Install a Linux Distribution from the Microsoft Store:

   • Choose and install your preferred Linux distribution (e.g., Ubuntu, Debian, Fedora).

3. Set Up Your Linux Distribution:

   • Launch the Linux distribution and complete the account setup.

4. Update and Upgrade Your Linux Distribution:

   • Run sudo apt update && sudo apt upgrade in the Linux terminal.

For a detailed WSL setup guide, view How to Set Up WSL on YouTube.

---

🎬 How to Use MocapNET V4

Preparing for Usage

Before running MocapNET V4, navigate to the appropriate directory:

cd src/python/mnet4

Quick Start with a Sample Video

Now you're ready to download and use a sample video:

1. Download a Sample Video:

   wget http://ammar.gr/mocapnet/shuffle.webm -O shuffle.webm
   

2. Process the Video: Execute the following command to run MocapNET on the downloaded video:

   python3 -m mediapipeHolisticWebcamMocapNET --from shuffle.webm --ik 0.001 99 99 --smooth 60 10 --all --save --plot --headless 2
   

Convert Output BVH

For further processing, such as generating a BVH file without face and toes bones, execute:

./GroundTruthDumper --from dependencies/RGBDAcquisition/opengl_acquisition_shared_library/opengl_depth_and_color_renderer/Motions/DAZFriendlyCGSPEED_ZXYAndHandsAxisBigHands.bvh --merge src/python/mnet4/out.bvh dependencies/RGBDAcquisition/opengl_acquisition_shared_library/opengl_depth_and_color_renderer/Motions//mergeDazFriendlyAndAddHead.profile --setPositionRotation 0 0 0 0 0 0 --bvh remade.bvh

---

Notes:

• The output bvh file will be stored inside this directory MocapNET/src/python/mnet4 with the filename out.bvh

• The converted bvh file will be stored in the root directory MocapNET with the filename remade.bvh

---

Command-Line Options Explained

• --headless: Skip the live video display. Ideal for headless servers or GUI-less environments.
• --live: Launches a live demo. Functionalities may vary.
• --mt: Enable multi-threading for better performance on multi-core CPUs.
• --calib [filename]: Use a specific calibration file for camera adjustments.
• --frameskip [number]: Skip a set number of frames to reduce load or speed up processing.
• --nnsubsample [number]: Run the neural network at set intervals (e.g., every 2nd frame).
• --ik [float int int]: Set parameters for inverse kinematics optimization.
• --smooth [float float]: Apply smoothing to motion capture data.
• --noik: Turn off inverse kinematics processing.
• --aspectCorrection [float]: Adjust video or image aspect ratio.
• --noise [float]: Add noise to input for testing under varied conditions.
• --size [int int]: Define video/image dimensions.
• --scale [float]: Scale the output motion data.
• --plot: Plot BVH channels for skeletal animation visualization.
• --all: Activate all processing features (body, eyes, mouth, hands).
• --nobody: Disable body joint estimation.
• --face: Enable face joint estimation.
• --eyes / --reye: Estimate right eye joints.
• --mouth: Enable mouth joint estimation.
• --hands: Estimate hand joints.
• --save: Save outputs (images, videos, data, logs).
• --engine [engine_name]: Choose a specific neural network backend.
• --from [filepath]: Specify input source (video file or live feed).
• --profile: Activate performance profiling.

THIS README IS A W.I.P