Journal Article

Robust skeletal motion tracking using temporal and spatial synchronization of two video streams

Gisleris, Ervinas, Abromavičius, Vytautas, Serackis, Artūras, Daunoravičienė, Kristina, Žižienė, Jurgita, Maskeliūnas, Rytis

2025

Overview

Accurate and reliable skeletal motion tracking is essential for rehabilitation monitoring, enabling objective assessment of patient progress and facilitating telerehabilitation applications. Traditional marker-based motion capture systems, while highly accurate, are costly and impractical for home rehabilitation, whereas marker-less methods often suffer from depth estimation errors and occlusions. Recent studies have explored various computer vision and deep learning approaches for human pose estimation, yet challenges remain in ensuring robust depth accuracy and tracking under occlusion conditions. This study proposes a three-dimensional human skeleton tracking system for upper limb activities that integrates temporal and spatial synchronization to improve depth estimation accuracy for rehabilitation exercises. The proposed system combines a 90° secondary camera to compensate for the depth prediction inaccuracies inherent in single-camera systems, reducing error margins by up to 0.4 m. In addition, a linear regression-based depth error correction model is implemented to refine depth coordinates, further improving tracking precision. The Kalman filtering framework is employed to enhance temporal consistency, allowing real-time interpolation of missing joint positions. Experimental results demonstrate that the proposed method significantly reduces depth estimation errors of the elbow and wrist joint ( p < 0.001) compared to single camera setups, particularly in scenarios involving occlusions and non-frontal perspectives. This study provides a cost-effective and scalable solution for remote patient monitoring and motor function evaluation.

• Publisher: Public Library of Science,Public Library of Science (PLoS)
• Subject: Accuracy / Algorithms / Biology and Life Sciences / Cameras / Computer vision / Datasets / Deep learning / Elbow / Elbow (anatomy) / Elbow Joint - physiology / Engineering and Technology / Error correction / Error reduction / Estimation accuracy / Estimation errors / Human mechanics / Human motion / Human skeleton / Humans / Interpolation / Joints (anatomy) / Kalman filters / Machine vision / Medicine and Health Sciences / Motion capture / Movement - physiology / Neural networks / Occlusion / Physical Sciences / Physiological aspects / Pose estimation / R&D / Real time / Regression analysis / Rehabilitation / Remote monitoring / Research & development / Research and Analysis Methods / Robustness (mathematics) / Sensors / Skeleton / Streaming media / Synchronism / Synchronization / Technology application / Tracking systems / Video data / Video Recording - methods / Virtual reality / Wrist / Wrist Joint - physiology