Journal Article

Multi-Camera-Based Human Activity Recognition for Human–Robot Collaboration in Construction

Jang, Youjin, Jeong, Inbae, Younesi Heravi, Moein, Sarkar, Sajib, Shin, Hyunkyu, Ahn, Yonghan

2023

Overview

As the use of construction robots continues to increase, ensuring safety and productivity while working alongside human workers becomes crucial. To prevent collisions, robots must recognize human behavior in close proximity. However, single, or RGB-depth cameras have limitations, such as detection failure, sensor malfunction, occlusions, unconstrained lighting, and motion blur. Therefore, this study proposes a multiple-camera approach for human activity recognition during human–robot collaborative activities in construction. The proposed approach employs a particle filter, to estimate the 3D human pose by fusing 2D joint locations extracted from multiple cameras and applies long short-term memory network (LSTM) to recognize ten activities associated with human and robot collaboration tasks in construction. The study compared the performance of human activity recognition models using one, two, three, and four cameras. Results showed that using multiple cameras enhances recognition performance, providing a more accurate and reliable means of identifying and differentiating between various activities. The results of this study are expected to contribute to the advancement of human activity recognition and utilization in human–robot collaboration in construction.

• Publisher: MDPI AG,MDPI
• Subject: Accuracy / Algorithms / Automation / Cameras / Cognition & reasoning / Collaboration / Construction industry / Deep learning / Efficiency / human activity recognition / Human acts / Human behavior / Human mechanics / human pose estimation / Humans / Lighting / long short-term memory / Motion / multiple cameras / particle filter / Productivity / Robotics - methods / Robotics industry / Robots / Sensors