Explore the vast range of titles available.

Vision-based grasping for mobile manipulators poses significant challenges in machine perception, computational efficiency, and real-world deployment. This study presents a computationally lightweight, end-to-end grasp detection framework that integrates object detection, object pose estimation, and grasp point prediction for a mobile manipulator equipped with a parallel gripper. A transformation model is developed to map coordinates from the image frame to the robot frame, enabling accurate manipulation. To evaluate system performance, a benchmark and a dataset tailored to pick-and-pack grocery tasks are introduced. Experimental validation demonstrates an average execution time of under 5 s on an edge device, achieving a 100% success rate on Level 1 and 96% on Level 2 of the benchmark. Additionally, the system achieves an average compute-to-speed ratio of 0.0130, highlighting its energy efficiency. The proposed framework offers a practical, robust, and efficient solution for lightweight robotic applications in real-world environments.

Mobile manipulators have become essential platforms for autonomous tasks that demand high-quality performance and efficient operational processes. This paper presents a complete grocery pick-and-pack system for a mobile manipulator, integrating a graphical user interface (GUI) with an end-to-end vision-based grasp detection pipeline designed for lightweight computation. The system is evaluated on the Grocery Pick-and-Pack Benchmark (Level-3), the most challenging level due to deformable objects, dimensional constraints, and strict grasp-point requirements. Experimental results demonstrate an average success rate of 92% across five item classes, with the deformable sweet bag the most challenging at 60% and an average execution time of 7.5 s on an edge device. The system achieves strong computational efficiency, reflected by a compute-to-speed ratio (CSR) of 0.008, with a total model size of only 30.9 MB. Performance is further validated across multiple hardware platforms and under real competition scenarios in the European Robotics League 2025. The findings highlight the practical impact of lightweight, vision-based mobile manipulation and provide insights into current challenges and future research directions for autonomous robotic applications.

Vision-based robotic grasping is a fundamental task in robotic control. Dexterous and precise grasp control of the robotic arm is challenging and a critical technique for the manufacturing and emerging robot service industry. Current state-of-art methods adopt RGB-D images or point clouds in an attempt to obtain an accurate, robust, and real-time policy. However, most of these methods only use single modal data or ignore the uncertainty of sampling data especially the depth information. Even they leverage multi-modal data, they seldom fuse the features in different scales. All of these results in unreliable grasp prediction inevitably. In this paper, we propose a novel multi-modal neural network to predict grasps in real-time. The key idea is to fuse RGB and depth information hierarchically and quantify the uncertainty of raw depth data to re-weight the depth features. For higher grasping performance, a background extraction module and depth re-estimation module are used to reduce the influence caused by the incompletion and low-quality of the raw data. We evaluate the performance on the Cornell Grasp Dataset and provide a series of extensive experiments to demonstrate the advantages of our method on a real robot. The results indicate the superiority of our proposed method by outperforming the state-of-the-art methods significantly in all metrics.

Tactile sensing is crucial for providing physical interaction between the robotic hand and objects, enabling effective and safe grasping and manipulation. Vision‐based tactile sensors have gained increasing popularity due to their high spatial resolution and extensibility. Currently, most vision‐based tactile sensors utilize either the marker array tracking method or the photometric stereo method. Spatial resolution of the marker array is constrained by the density of markers, while the photometric stereo approach requires complex illumination. Herein, we propose a miniature vision‐based tactile sensor exploiting a speckle skin, namely, SpeckleTac. By applying a dense optical flow method with adaptive Kalman filtering and dynamic weight fusion, high‐resolution and accurate 2D displacement fields are obtained, and subsequently the reconstruction of the 3D surface. Then, contact object classification and pose estimation are realized through a ResNet‐based architecture, SPTacNet, integrated with squeeze‐and‐excitation (SE) residual blocks. Furthermore, by incorporating a TV‐regularized iFEM method, contact pressure distribution and 6‐axis force and torque are obtained. We demonstrate that the SpeckleTac can accurately locate the centroid of an unknown object, enabling stable grasping and facilitating safe handling of blood samples in glass tubes. This work provides a promising solution for multimodal tactile perception of robotic hands in “real‐world” applications. SpeckleTac, a miniature vision‐based tactile sensor, utilizes a speckle‐pattern skin and optical flow‐based scalable virtual marker tracking. Combined with advanced algorithms, it achieves high‐resolution 3D surface reconstruction, precise contact perception, and stable grasping capabilities. Additionally, it enables safe handling of fragile samples, offering a practical and reliable solution for multimodal tactile perception for robotic hands in real‐world scenarios.

In this study, we present a novel algorithm aimed at detecting the grasping poses of target objects in the visual grasping process of robotic arms, specifically addressing challenges posed by insufficient lighting and low exposure. The proposed algorithm comprises of two key components: an image enhancement network designed for low-light scenes, and an object planar grasping pose detection network. To enhance the quality of images captured in low-light conditions, a comprehensive end-to-end image processing model named Multi-scale Context Residual Convolutional Network (MCRNet) is introduced. The model not only learns the features and representations of images, but also enhances its brightness, contrast, and details. It improves the overall enhancement effectiveness of low-light image. The enhanced images are subsequently fed into the Context Residual Convolutional Network (CRNet), which extracts crucial visual features and detects object planar grasping poses. To assess the performance of our approach, RGB-depth sensors are employed on two widely-used datasets: the Cornell Grasping Dataset and the Jacquard Grasping Dataset. The proposed algorithm achieves the grasping success rates of 93.22% and 90.22%, respectively. Furthermore, the effectiveness of our system in real-world low-light environments is validated by conducting robotic arm grasping experiments, the impressive grasping accuracy of 86% is achieved. Compared to scenarios without image enhancement, the proposed MCRNet low-light enhancement network significantly improves the grasp success rate of robotic arms operating under insufficient lighting conditions.

Limited memory and resources frequently hinder real-time detection and pose solving for space robot grasping in unmanned deep space exploration equipment. To achieve a high accuracy, high speed, and robust robotic grasping posture detection system under the condition of low parameters, a lightweight network for robotic visual grasp based on pixel-level detection is proposed. (1) To simplify the spatial complexity of the network and improve detection accuracy, a GS-net was employed. Grouped dot convolution reduces the number of parameters in the convolutional layer, and instance normalization performs independent segmentation for each convolutional layer of feature map on a per-image and per-channel basis, which speeds up the convergence of network and enhances detection precision. Subsequently, the SE module assigns weights to each feature channel of the feature maps outputted by the grouped dot convolution, in order to extract feature information with high weights. (2) To address the issue of edge information loss during upsampling and enhance the detection accuracy, a method involving bilinear interpolation of features between downsampling and upsampling is employed. By performing bilinear interpolation on feature maps of the same dimension between downsampling and upsampling, the resolution of the output feature maps is increased, which, in turn, further improves the detection accuracy. On the selected Cornell data test set, the network trained in this paper achieves a detection accuracy of 96.6%, with a detection speed of 25.3 ms, and the network model parameters are 0.365 million. Additionally, to extend robotic grasp detection into the space, we created our own satellite model and designed corresponding experiments to verify detection performance under different environmental conditions. Ultimately, PyBullet simulation experiments on the Dex-Net 2.0 dataset demonstrated that the research can be effectively applied to practical applications.

Soft robotic grippers provide compliant and adaptive manipulation, but most existing designs address actuation speed, adaptability, modularity, or sensing individually rather than in combination. This paper presents an electro-actuated customizable stacked Fin Ray gripper that integrates these capabilities within a single design. The gripper employs a compact solenoid for fast grasping, multiple vertically stacked Fin Ray segments for improved 3D conformity, and interchangeable silicone or TPU fins that can be tuned for task-specific stiffness and geometry. In addition, a light-guided, vision-based sensing approach is introduced to capture deformation without embedded sensors. Experimental studies—including free-fall object capture and optical shape sensing—demonstrate rapid solenoid-driven actuation, adaptive grasping behavior, and clear visual detectability of fin deformation. Complementary simulations using Cosserat-rod modeling and bond-graph analysis characterize the deformation mechanics and force response. Overall, the proposed gripper provides a practical soft-robotic solution that combines speed, adaptability, modular construction, and straightforward sensing for diverse object-handling scenarios.

Approximately half of the patients with spinal cord injuries in Japan have a cervical spinal cord injury. Owing to the trunk dysfunction, patients with high-level spinal cord injuries have particular difficulty when searching for or picking up objects from the floor. Recently, welfare robot arms have been developed to help such individuals increase self-reliance. In this study, we propose an operating system that includes an eye-in-hand system with a touchscreen interface for grasping objects from the floor and delivering them to the individual. In the proposed method, the visual information of the target object is shown on a touchscreen interface. The patient specifies the target position for the robot arm by drawing a line on the target object on the interface. We conducted an experiment to compare the proposed interface with an on-screen joystick to demonstrate the proposed system’s efficiency and its ability to reduce physical burden. The results show that the proposed method is both quicker to use and effectively reduces the physical burden on the user compared to the conventional method.