Explore the vast range of titles available.

This article identifies and summarizes software tools and frameworks proposed in the Human–Robot Interaction (HRI) literature for developing extended reality (XR) experiences using game engines. This review includes primary studies proposing Virtual Reality (VR), Augmented Reality (AR), and Mixed Reality (MR) solutions where humans can control or interact with real robotic platforms using devices that extend the user’s reality. The objective of this article is not to present an extensive list of applications and tools. Instead, we present recent, relevant, common, and accessible frameworks and software tools implemented in research articles published in high-impact robotics conferences and journals. For this, we searched papers published during a seven-years period between 2015 and 2022 in relevant databases for robotics (Science Direct, IEEE Xplore, ACM digital library, Springer Link, and Web of Science). Additionally, we present and classify the application context of the reviewed articles in four groups: social robotics, programming of industrial robots, teleoperation of industrial robots, and Human–Robot collaboration (HRC).

This study presents an error recovery architecture for future variable-mix variable-volume production based on cyber-physical-human systems (CPHS). It focuses on bin picking, which is a crucial manufacturing process for handling bulk industrial parts during kitting. One of the main challenges in bin picking is efficiently introducing perturbations to arbitrarily placed parts and make all parts graspable and resolve deadlock situations. For example, a suction-type gripper is advantageous for handling objects stably without geometric models as it can easily adhere to flat surfaces. However, the success rate of bin picking using a suction gripper depends on the orientation of the target part. If its flat graspable surface does not face upward, the suction gripper cannot attach to and pick up the target object, resulting in a deadlock. In this case, an external force must be applied to change the orientation of the target object to resume the bin-picking process. A conventional, albeit inefficient, solution is a human worker or an additional mechanism that perturbs the container. Because applying such a perturbation by a versatile robot is challenging due to the limited physical information, a promising approach for efficient error recovery is a combination of human remote instruction and automated trajectory planning. This study developed a CPHS-based architecture to facilitate error recovery through smooth human-robot collaboration. We perform three experiments to demonstrate the feasibility of this approach for efficient error recovery.

In this paper, toward robotic gait assistance, we investigate the feasibility of a cane-type assistive mobile robot accompanying the user autonomously through a clinical pilot experiment. As widely known, gait ability is important for all people to keep their quality of life. However, for people having weakened lower limbs such as elderly people, their postural sway during walking could be insuppressible and cause falling. To support the gait motion of elderly people, our group has been developing a series of hand-holding cane robots named Intelligent Cane. Such assistive robots are expected to remove barriers to the independent lives of elderly people. Recently, we have focused on the potential of a companion robot that follows the user ahead and can be touched or grasped for bracing whenever the user needs it. In order to demonstrate proof of our concept through an experiment with a motion capture system, we propose a user companion strategy that enables our cane robot to keep a constant relative distance between the robot and the user walking on a treadmill. We evaluate the accuracy of the user companion in an experiment where a user walks on a treadmill. Then, we conduct a clinical experiment with three healthy subjects walking on with the treadmill with our cane robot as a pilot study. Through the clinical experiment, we evaluate a postural stabilization effect of physical interaction with the robot and discuss the feasibility of our robotic gait assistance methodology.

The application of teleoperation to control robotic arms has been widely explored, and user-friendly teleoperation systems have been studied for facilitating higher performance and lower operational burden. To investigate the dominant factors in a practical teleoperation system, this study focused on the characteristics of an interface used to operate a robotic arm. The usability of an interface depends on the characteristics of the manipulation tasks to be completed; however, systematic comparisons of different interfaces across different tasks remain limited. In this study, we compared two widely used teleoperation interfaces, a 3D mouse and a VR controller, for two simple yet broadly applicable tasks with a six-degree-of-freedom (6DoF) robotic arm: repetitively pushing buttons and rotating knobs. Participants (N = 23) controlled a robotic arm with 6DoF to push buttons and rotate knobs as many times as possible in 3-minute trials. Each trial was followed by a NASA-TLX workload rating. The results showed a clear connection between the interface and task performance: the VR controller yielded higher performance for pushing buttons, whereas the 3D mouse performed better and was less demanding for knob rotation. These findings highlight the importance of considering dominant motion primitives of the task when designing practical teleoperation interfaces.

In this study, we introduce an optimization framework aimed at enhancing the efficiency of motion priority design in scenarios involving automated and teleoperated robots within an industrial recovery context. The escalating utilization of industrial robots at manufacturing sites has been instrumental in mitigating human workload. Nevertheless, the challenge persists in achieving effective human-robot collaboration/cooperation where human workers and robots share a workspace for collaborative tasks. In the event of an industrial robot encountering a failure, it necessitates the suspension of the corresponding factory cell for safe recovery. Given the limited capacity of pre-programmed robots to rectify such failures, human intervention becomes imperative, requiring entry into the robot workspace to address the dropped object while the robot system is halted. This non-continuous manufacturing process results in productivity loss. Robotic teleoperation has emerged as a promising technology enabling human workers to undertake high-risk tasks remotely and safely. Our study advocates for the incorporation of robotic teleoperation in the recovery process during manufacturing failure scenarios, which is referred to as \"Cooperative Tele-Recovery\". Our proposed approach involves the formulation of priority rules designed to facilitate collision avoidance between manufacturing and recovery robots. This, in turn, ensures a continuous manufacturing process with minimal production loss within a configurable risk limitation. We present a comprehensive motion priority optimization framework, encompassing an HRC simulator-based priority optimization and a cooperative multi-robot controller, to identify optimal parameters for the priority function. The framework dynamically adjusts the allocation of motion priorities for manufacturing and recovery robots while adhering to predefined risk limitations.

In this study, we introduce an optimization framework aimed at enhancing the efficiency of motion priority design in scenarios involving automated and teleoperated robots within an industrial recovery context. The escalating utilization of industrial robots at manufacturing sites has been instrumental in mitigating human workload. Nevertheless, the challenge persists in achieving effective human-robot collaboration/cooperation where human workers and robots share a workspace for collaborative tasks. In the event of an industrial robot encountering a failure, it necessitates the suspension of the corresponding factory cell for safe recovery. Given the limited capacity of pre-programmed robots to rectify such failures, human intervention becomes imperative, requiring entry into the robot workspace to address the dropped object while the robot system is halted. This non-continuous manufacturing process results in productivity loss. Robotic teleoperation has emerged as a promising technology enabling human workers to undertake high-risk tasks remotely and safely. Our study advocates for the incorporation of robotic teleoperation in the recovery process during manufacturing failure scenarios, which is referred to as \"Cooperative Tele-Recovery\". Our proposed approach involves the formulation of priority rules designed to facilitate collision avoidance between manufacturing and recovery robots. This, in turn, ensures a continuous manufacturing process with minimal production loss within a configurable risk limitation. We present a comprehensive motion priority optimization framework, encompassing an HRC simulator-based priority optimization and a cooperative multi-robot controller, to identify optimal parameters for the priority function. The framework dynamically adjusts the allocation of motion priorities for manufacturing and recovery robots while adhering to predefined risk limitations.

This paper presents two types of extended diffeomorphism designs to compensate for spatial placement differences between robot workspaces. Teleoperation of multiple robots is attracting attention to expand the utilization of the robot embodiment. Real-time reproduction of robot motion would facilitate the efficient execution of similar tasks by multiple robots. A challenge in the motion reproduction is compensating for the spatial arrangement errors of target keypoints in robot workspaces. This paper proposes a methodology for smooth mappings that transform primary robot poses into follower robot poses based on the predefined key points in each workspace. Through a picking task experiment using a dual-arm UR5 robot, this study demonstrates that the proposed mapping generation method can balance lower mapping errors for precise operation and lower mapping gradients for smooth replicated movement.