Technical Name Lightweight Real-Time 3D Human Motion Capture System Based on Multi-View RGB Cameras
Project Operator National Tsing Hua University
Project Host 胡敏君
Summary
Our lightweight motion capture system focuses on lightweight deployment, easy operation, and fast execution. Unlike traditional expensive and complex motion capture systems, it uses consumer-grade action cameras and a temporal correlation-based architecture to achieve accurate human pose estimation. This enables flexible, fast, and location-independent motion capture, allowing non-professional league teams or general groups to achieve professional and scientific sports training at a lower cost.
Scientific Breakthrough
Existing 2D human pose estimation models are accurate but need long inference times, with fps dropping significantly in multi-person or multi-view videos. To address this, we use a GRU architecture to combine the large model's accuracy with the small model's fast inference. The GRU then uses the large model's results to correct the small model's estimates through temporal correlations, enabling our system to estimate each person's pose in real-time, even in multi-person videos.
Industrial Applicability
Our motion capture system can reconstruct 3D human skeletons and, with different 2D object detection models, capture 3D positions of other objects. In baseball, for example, it can detect the 3D coordinates of the body, bat and ball, allowing coaches to view player movement from any angle as many times as needed for posture analysis. In short, our technology is suitable for any situation requiring human pose recognition, offering low-cost, high-precision, lightweight motion capture and analysis.
Keyword Multi-Camera Control System Multi-Camera Synchronization and Calibration 2D Human Pose Estimation Fusion Model 3D Pose Reconstruction and Visualization Lightweight Motion Capture System
Notes
  • Contact
  • Min-Chun Hu