Explore the vast range of titles available.

Accurate and automatic identification of tree species information at the individual tree scale is of great significance for fine-scale investigation and management of forest resources and scientific assessment of forest ecosystems. Despite the fact that numerous studies have been conducted on the delineation of individual tree crown and species classification using drone high-resolution red, green and blue (RGB) images, and Light Detection and Ranging (LiDAR) data, performing the above tasks simultaneously has rarely been explored, especially in complex forest environments. In this study, we improve upon the state of the Mask region-based convolution neural network (Mask R-CNN) with our proposed attention complementary network (ACNet) and edge detection R-CNN (ACE R-CNN) for individual tree species identification in high-density and complex forest environments. First, we propose ACNet as the feature extraction backbone network to fuse the weighted features extracted from RGB images and canopy height model (CHM) data through an attention complementary module, which is able to selectively fuse weighted features extracted from RGB and CHM data at different scales, and enables the network to focus on more effective information. Second, edge loss is added to the loss function to improve the edge accuracy of the segmentation, which is calculated through the edge detection filter introduced in the Mask branch of Mask R-CNN. We demonstrate the performance of ACE R-CNN for individual tree species identification in three experimental areas of different tree species in southern China with precision (P), recall (R), F1-score, and average precision (AP) above 0.9. Our proposed ACNet–the backbone network for feature extraction–has better performance in individual tree species identification compared with the ResNet50-FPN (feature pyramid network). The addition of the edge loss obtained by the Sobel filter further improves the identification accuracy of individual tree species and accelerates the convergence speed of the model training. This work demonstrates the improved performance of ACE R-CNN for individual tree species identification and provides a new solution for tree-level species identification in complex forest environments, which can support carbon stock estimation and biodiversity assessment.

High-resolution UAV imagery paired with a convolutional neural network approach offers significant advantages in accurately measuring forestry ecosystems. Despite numerous studies existing for individual tree crown delineation, species classification, and quantity detection, the comprehensive situation in performing the above tasks simultaneously has rarely been explored, especially in mixed forests. In this study, we propose a new method for individual tree segmentation and identification based on the improved Mask R-CNN. For the optimized network, the fusion type in the feature pyramid network is modified from down-top to top-down to shorten the feature acquisition path among the different levels. Meanwhile, a boundary-weighted loss module is introduced to the cross-entropy loss function Lmask to refine the target loss. All geometric parameters (contour, the center of gravity and area) associated with canopies ultimately are extracted from the mask by a boundary segmentation algorithm. The results showed that F1-score and mAP for coniferous species were higher than 90%, and that of broadleaf species were located between 75–85.44%. The producer’s accuracy of coniferous forests was distributed between 0.8–0.95 and that of broadleaf ranged in 0.87–0.93; user’s accuracy of coniferous was distributed between 0.81–0.84 and that of broadleaf ranged in 0.71–0.76. The total number of trees predicted was 50,041 for the entire study area, with an overall error of 5.11%. The method under study is compared with other networks including U-net and YOLOv3. Results in this study show that the improved Mask R-CNN has more advantages in broadleaf canopy segmentation and number detection.

In orchard fruit picking systems for pears, the challenge is to identify the full shape of the soft fruit to avoid injuries while using robotic or automatic picking systems. Advancements in computer vision have brought the potential to train for different shapes and sizes of fruit using deep learning algorithms. In this research, a fruit recognition method for robotic systems was developed to identify pears in a complex orchard environment using a 3D stereo camera combined with Mask Region-Convolutional Neural Networks (Mask R-CNN) deep learning technology to obtain targets. This experiment used 9054 RGBA original images (3018 original images and 6036 augmented images) to create a dataset divided into a training, validation, and testing sets. Furthermore, we collected the dataset under different lighting conditions at different times which were high-light (9–10 am) and low-light (6–7 pm) conditions at JST, Tokyo Time, August 2021 (summertime) to prepare training, validation, and test datasets at a ratio of 6:3:1. All the images were taken by a 3D stereo camera which included PERFORMANCE, QUALITY, and ULTRA models. We used the PERFORMANCE model to capture images to make the datasets; the camera on the left generated depth images and the camera on the right generated the original images. In this research, we also compared the performance of different types with the R-CNN model (Mask R-CNN and Faster R-CNN); the mean Average Precisions (mAP) of Mask R-CNN and Faster R-CNN were compared in the same datasets with the same ratio. Each epoch in Mask R-CNN was set at 500 steps with total 80 epochs. And Faster R-CNN was set at 40,000 steps for training. For the recognition of pears, the Mask R-CNN, had the mAPs of 95.22% for validation set and 99.45% was observed for the testing set. On the other hand, mAPs were observed 87.9% in the validation set and 87.52% in the testing set using Faster R-CNN. The different models using the same dataset had differences in performance in gathering clustered pears and individual pear situations. Mask R-CNN outperformed Faster R-CNN when the pears are densely clustered at the complex orchard. Therefore, the 3D stereo camera-based dataset combined with the Mask R-CNN vision algorithm had high accuracy in detecting the individual pears from gathered pears in a complex orchard environment.

Tropical forests are a major component of the global carbon cycle and home to two-thirds of terrestrial species. Upper-canopy trees store the majority of forest carbon and can be vulnerable to drought events and storms. Monitoring their growth and mortality is essential to understanding forest resilience to climate change, but in the context of forest carbon storage, large trees are underrepresented in traditional field surveys, so estimates are poorly constrained. Aerial photographs provide spectral and textural information to discriminate between tree crowns in diverse, complex tropical canopies, potentially opening the door to landscape monitoring of large trees. Here we describe a new deep convolutional neural network method, Detectree2, which builds on the Mask R-CNN computer vision framework to recognize the irregular edges of individual tree crowns from airborne RGB imagery. We trained and evaluated this model with 3797 manually delineated tree crowns at three sites in Malaysian Borneo and one site in French Guiana. As an example application, we combined the delineations with repeat lidar surveys (taken between 3 and 6 years apart) of the four sites to estimate the growth and mortality of upper-canopy trees. Detectree2 delineated 65 000 upper-canopy trees across 14 km2 of aerial images. The skill of the automatic method in delineating unseen test trees was good (F1 score = 0.64) and for the tallest category of trees was excellent (F1 score = 0.74). As predicted from previous field studies, we found that growth rate declined with tree height and tall trees had higher mortality rates than intermediate-size trees. Our approach demonstrates that deep learning methods can automatically segment trees in widely accessible RGB imagery. This tool (provided as an open-source Python package) has many potential applications in forest ecology and conservation, from estimating carbon stocks to monitoring forest phenology and restoration.Python package available to install at https://github.com/PatBall1/ Detectree2.

In this paper, a manipulation planning method for object re-orientation based on semantic segmentation keypoint detection is proposed for robot manipulator which is able to detect and re-orientate the randomly placed objects to a specified position and pose. There are two main parts: (1) 3D keypoint detection system; and (2) manipulation planning system for object re-orientation. In the 3D keypoint detection system, an RGB-D camera is used to obtain the information of the environment and can generate 3D keypoints of the target object as inputs to represent its corresponding position and pose. This process simplifies the 3D model representation so that the manipulation planning for object re-orientation can be executed in a category-level manner by adding various training data of the object in the training phase. In addition, 3D suction points in both the object’s current and expected poses are also generated as the inputs of the next operation stage. During the next stage, Mask Region-Convolutional Neural Network (Mask R-CNN) algorithm is used for preliminary object detection and object image. The highest confidence index image is selected as the input of the semantic segmentation system in order to classify each pixel in the picture for the corresponding pack unit of the object. In addition, after using a convolutional neural network for semantic segmentation, the Conditional Random Fields (CRFs) method is used to perform several iterations to obtain a more accurate result of object recognition. When the target object is segmented into the pack units of image process, the center position of each pack unit can be obtained. Then, a normal vector of each pack unit’s center points is generated by the depth image information and pose of the object, which can be obtained by connecting the center points of each pack unit. In the manipulation planning system for object re-orientation, the pose of the object and the normal vector of each pack unit are first converted into the working coordinate system of the robot manipulator. Then, according to the current and expected pose of the object, the spherical linear interpolation (Slerp) algorithm is used to generate a series of movements in the workspace for object re-orientation on the robot manipulator. In addition, the pose of the object is adjusted on the z-axis of the object’s geodetic coordinate system based on the image features on the surface of the object, so that the pose of the placed object can approach the desired pose. Finally, a robot manipulator and a vacuum suction cup made by the laboratory are used to verify that the proposed system can indeed complete the planned task of object re-orientation.

Core samples from trees are a critical reservoir of ecological information, informing our understanding of past climates, as well as contemporary ecosystem responses to global change. Manual measurements of annual growth rings in trees are slow, labour‐intensive and subject to human bias, hindering the generation of big datasets. We present an alternative, neural network‐based implementation that automates detection and measurement of tree‐ring boundaries from coniferous species. We trained our Mask R‐CNN extensively on over 8000 manually annotated ring boundaries from microscope‐imaged Norway Spruce Picea abies increment cores. We assessed the performance of the trained model after post‐processing on real‐world data generated from our core processing pipeline. The CNN after post‐processing performed well, with recognition of over 98% of ring boundaries (recall) with a precision in detection of 96% when tested on real‐world data. Additionally, we have implemented automatic measurements based on minimum distance between rings. With minimal editing for missed ring detections, these measurements were 98% correlated with human measurements of the same samples. Tests on other three conifer species demonstrate that the CNN generalizes well to other species with similar structure. We demonstrate the efficacy of automating the measurement of growth increment in tree core samples. Our CNN‐based system provides high predictive performance in terms of both tree‐ring detection and growth rate determination. Our application is readily deployable as a Docker container and requires only basic command line skills. Additionally, an easy re‐training option allows users to expand capabilities to other wood types. Application outputs include both editable annotations of predictions as well as ring‐width measurements in a commonly used .pos format, facilitating the efficient generation of large ring‐width measurement datasets from increment core samples, an important source of environmental data.

Over a decade ago, the formation of neutrophil extracellular traps (NETs) was described as a novel mechanism employed by neutrophils to tackle infections. Currently applied methods for NETs release quantification are often limited by the use of unspecific dyes and technical difficulties. Therefore, we aimed to develop a fully automatic image processing method for the detection and quantification of NETs based on live imaging with the use of DNA-staining dyes. For this purpose, we adopted a recently proposed Convolutional Neural Network (CNN) model called Mask R-CNN. The adopted model detected objects with quality comparable to manual counting—Over 90% of detected cells were classified in the same manner as in manual labelling. Furthermore, the inhibitory effect of GW 311616A (neutrophil elastase inhibitor) on NETs release, observed microscopically, was confirmed with the use of the CNN model but not by extracellular DNA release measurement. We have demonstrated that a modern CNN model outperforms a widely used quantification method based on the measurement of DNA release and can be a valuable tool to quantitate the formation process of NETs.

The detection of road facilities or roadside structures is essential for high-definition (HD) maps and intelligent transportation systems (ITSs). With the rapid development of deep-learning algorithms in recent years, deep-learning-based object detection techniques have provided more accurate and efficient performance, and have become an essential tool for HD map reconstruction and advanced driver-assistance systems (ADASs). Therefore, the performance evaluation and comparison of the latest deep-learning algorithms in this field is indispensable. However, most existing works in this area limit their focus to the detection of individual targets, such as vehicles or pedestrians and traffic signs, from driving view images. In this study, we present a systematic comparison of three recent algorithms for large-scale multi-class road facility detection, namely Mask R-CNN, YOLOx, and YOLOv7, on the Mapillary dataset. The experimental results are evaluated according to the recall, precision, mean F1-score and computational consumption. YOLOv7 outperforms the other two networks in road facility detection, with a precision and recall of 87.57% and 72.60%, respectively. Furthermore, we test the model performance on our custom dataset obtained from the Japanese road environment. The results demonstrate that models trained on the Mapillary dataset exhibit sufficient generalization ability. The comparison presented in this study aids in understanding the strengths and limitations of the latest networks in multiclass object detection on large-scale street-level datasets.

As outdoor activities are necessary for maintaining our health, research interest in environmental conditions such as the weather, atmosphere, and ultraviolet (UV) radiation is increasing. In particular, UV radiation, which can benefit or harm the human body depending on the degree of exposure, is recognized as an essential environmental factor that needs to be identified. However, unlike the weather and atmospheric conditions, which can be identified to some extent by the naked eye, UV radiation corresponds to wavelength bands that humans cannot recognize; hence, the intensity of UV radiation cannot be measured. Recently, although devices and sensors that can measure UV radiation have been launched, it is very difficult for ordinary users to acquire ambient UV radiation information directly because of the cost and inconvenience caused by operating separate devices. Herein, a deep neural network (DNN)-based ultraviolet index (UVI) calculation method is proposed using representative color information of sun object images. First, Mask-region-based convolutional neural networks (R-CNN) are applied to sky images to extract sun object regions and then detect the representative color of the sun object regions. Then, a deep learning model is constructed to calculate the UVI by inputting RGB color values, which are representative colors detected later along with the altitude angle and azimuth of the sun at that time. After selecting each day of spring and autumn, the performance of the proposed method was tested, and it was confirmed that accurate UVI could be calculated within a range of mean absolute error of 0.3.

In this paper, a fast wind turbine defect detection model is proposed with a Cascade Mask region Convolutional Neural network (Cascade Mask R-CNN). Instead of standard convolution in the backbone network of Cascade Mask R-CNN, a depthwise separable convolution is used to minimize the computation cost. Moreover, image augmentation and transfer learning techniques are also involved to enhance the performance of the proposed model. The detection and instance segmentation performance of the proposed model is compared with existing techniques in terms of mean intersection over union (MIoU), mean average precision (MAP) and classifier accuracy. The experimental results show that the proposed WTB defect detection and classification model shows better performance with 82.42% MAP, 87.49% MIoU and 97.8% classifier accuracy.