Explore the vast range of titles available.

Deep Residual Networks have recently been shown to significantly improve the performance of neural networks trained on ImageNet, with results beating all previous methods on this dataset by large margins in the image classification task. However, the meaning of these impressive numbers and their implications for future research are not fully understood yet. In this survey, we will try to explain what Deep Residual Networks are, how they achieve their excellent results, and why their successful implementation in practice represents a significant advance over existing techniques. We also discuss some open questions related to residual learning as well as possible applications of Deep Residual Networks beyond ImageNet. Finally, we discuss some issues that still need to be resolved before deep residual learning can be applied on more complex problems.

Intelligent machinery fault diagnosis system has been receiving increasing attention recently due to the potential large benefits of maintenance cost reduction, enhanced operation safety and reliability. This paper proposes a novel deep learning method for rotating machinery fault diagnosis. Since accurately labeled data are usually difficult to obtain in real industries, data augmentation techniques are proposed to artificially create additional valid samples for model training, and the proposed method manages to achieve high diagnosis accuracy with small original training dataset. Two augmentation methods are investigated including sample-based and dataset-based methods, and five augmentation techniques are considered in general, i.e. additional Gaussian noise, masking noise, signal translation, amplitude shifting and time stretching. The effectiveness of the proposed method is validated by carrying out experiments on two popular rolling bearing datasets. Fairly high diagnosis accuracy up to 99.9% can be obtained using limited training data. By comparing with the latest advanced researches on the same datasets, the superiority of the proposed method is demonstrated. Furthermore, the diagnostic performance of the deep neural network is extensively evaluated with respect to data augmentation strength, network depth and so forth. The results of this study suggest that the proposed intelligent fault diagnosis method offers a new and promising approach.

Deep learning plays a key role in the recent developments of machine learning. This paper develops a deep residual neural network (ResNet) for the regression of nonlinear functions. Convolutional layers and pooling layers are replaced by fully connected layers in the residual block. To evaluate the new regression model, we train and test neural networks with different depths and widths on simulated data, and we find the optimal parameters. We perform multiple numerical tests of the optimal regression model on multiple simulated data, and the results show that the new regression model behaves well on simulated data. Comparisons are also made between the optimal residual regression and other linear as well as nonlinear approximation techniques, such as lasso regression, decision tree, and support vector machine. The optimal residual regression model has better approximation capacity compared to the other models. Finally, the residual regression is applied into the prediction of a relative humidity series in the real world. Our study indicates that the residual regression model is stable and applicable in practice.

Automatically recognizing the modulation of radar signals is a necessary survival technique in electronic intelligence systems. In order to avoid the complex process of the feature extracting and realize the intelligent modulation recognition of various radar signals under low signal-to-noise ratios (SNRs), this paper proposes a method based on intrapulse signatures of radar signals using adaptive singular value reconstruction (ASVR) and deep residual learning. Firstly, the time-frequency spectrums of radar signals under low SNRs are improved after ASVR denoising processing. Secondly, a series of image processing techniques, including binarizing and morphologic filtering, are applied to suppress the background noise in the time-frequency distribution images (TFDIs). Thirdly, the training process of the residual network is achieved using TFDIs, and classification under various conditions is realized using the new-trained network. Simulation results show that, for eight kinds of modulation signals, the proposed approach still achieves an overall probability of successful recognition of 94.1% when the SNR is only −8 dB. Outstanding performance proves the superiority and robustness of the proposed method.

Real-time video objects detection, tracking, and recognition are challenging issues due to the real-time processing requirements of the machine learning algorithms. In recent years, video processing is performed by deep learning (DL) based techniques that achieve higher accuracy but require higher computations cost. This paper presents a recent survey of the state-of-the-art DL platforms and architectures used for deep vision systems. It highlights the contributions and challenges from over numerous research studies. In particular, this paper first describes the architecture of various DL models such as AutoEncoders, deep Boltzmann machines, convolution neural networks, recurrent neural networks and deep residual learning. Next, deep real-time video objects detection, tracking and recognition studies are highlighted to illustrate the key trends in terms of cost of computation, number of layers and the accuracy of results. Finally, the paper discusses the challenges of applying DL for real-time video processing and draw some directions for the future of DL algorithms.

Seismic data reconstruction is an essential process to reduce the effects of missing traces in field acquisition. Imputation and interpolation techniques have been proposed to reconstruct the subtle missing features, which is a challenging task that remains to be solved, such as the recovery capability for weak reflections and consecutively missing parts and the computational efficiency. In recent years, deep learning (DL), especially the residual network (ResNet), has gained remarkable success in seismic data reconstruction because it can precisely extract seismic features. In this overview, we probe into the residual module architecture and discuss in-depth the characteristics of the state-of-the-art residual modules, mainly including the split-transform-merge and the squeeze-and-attention. We use the detailed ablation experiments to investigate the roles of several key hyperparameters for each residual-based model. We explore the mechanism of the residual module’s effectiveness based on extensive qualitative and quantitative comparisons involving various residual networks and state-of-the-art models for seismic data reconstruction. The irregularly sampled and consecutively sampled scenarios demonstrate that the reconstruction performance of the residual modules is promising and superior to that of state-of-the-art deep networks.

Background. Alzheimer’s is a disease for which there is no cure. Diagnosing Alzheimer’s disease (AD) early facilitates family planning and cost control. The purpose of this study is to predict the presence of AD using socio-demographic, clinical, and magnetic resonance imaging (MRI) data. Early detection of AD enables family planning and may reduce costs by delaying long-term care. Accurate, non-imagery methods also reduce patient costs. The Open Access Series of Imaging Studies (OASIS-1) cross-sectional MRI data were analyzed. A gradient boosted machine (GBM) predicted the presence of AD as a function of gender, age, education, socioeconomic status (SES), and a mini-mental state exam (MMSE). A residual network with 50 layers (ResNet-50) predicted the clinical dementia rating (CDR) presence and severity from MRI’s (multi-class classification). The GBM achieved a mean 91.3% prediction accuracy (10-fold stratified cross validation) for dichotomous CDR using socio-demographic and MMSE variables. MMSE was the most important feature. ResNet-50 using image generation techniques based on an 80% training set resulted in 98.99% three class prediction accuracy on 4139 images (20% validation set) at Epoch 133 and nearly perfect multi-class predication accuracy on the training set (99.34%). Machine learning methods classify AD with high accuracy. GBM models may help provide initial detection based on non-imagery analysis, while ResNet-50 network models might help identify AD patients automatically prior to provider review.

With the rise of artificial intelligence, sensor-based human activity recognition (S-HAR) is increasingly being employed in healthcare monitoring for the elderly, fitness tracking, and patient rehabilitation using smart devices. Inertial sensors have been commonly used for S-HAR, but wearable devices have been demanding more comfort and flexibility in recent years. Consequently, there has been an effort to incorporate stretch sensors into S-HAR with the advancement of flexible electronics technology. This paper presents a deep learning network model, utilizing aggregation residual transformation, that can efficiently extract spatial–temporal features and perform activity classification. The efficacy of the suggested model was assessed using the w-HAR dataset, which included both inertial and stretch sensor data. This dataset was used to train and test five fundamental deep learning models (CNN, LSTM, BiLSTM, GRU, and BiGRU), along with the proposed model. The primary objective of the w-HAR investigations was to determine the feasibility of utilizing stretch sensors for recognizing human actions. Additionally, this study aimed to explore the effectiveness of combining data from both inertial and stretch sensors in S-HAR. The results clearly demonstrate the effectiveness of the proposed approach in enhancing HAR using inertial and stretch sensors. The deep learning model we presented achieved an impressive accuracy of 97.68%. Notably, our method outperformed existing approaches and demonstrated excellent generalization capabilities.

Detecting changes in land use and land cover (LULC) from space has long been the main goal of satellite remote sensing (RS), yet the existing and available algorithms for cloud classification are not reliable enough to attain this goal in an automated fashion. Clouds are very strong optical signals that dominate the results of change detection if they are not removed completely from imagery. As various architectures of deep learning (DL) have been proposed and advanced quickly, their potential in perceptual tasks has been widely accepted and successfully applied to many fields. A comprehensive survey of DL in RS has been reviewed, and the RS community has been suggested to be leading researchers in DL. Based on deep residual learning, semantic image segmentation, and the concept of atrous convolution, we propose a new DL architecture, named CloudNet, with an enhanced capability of feature extraction for classifying cloud and haze from Sentinel-2 imagery, with the intention of supporting automatic change detection in LULC. To ensure the quality of the training dataset, scene classification maps of Taiwan processed by Sen2cor were visually examined and edited, resulting in a total of 12,769 sub-images with a standard size of 224 × 224 pixels, cut from the Sen2cor-corrected images and compiled in a trainset. The data augmentation technique enabled CloudNet to have stable cirrus identification capability without extensive training data. Compared to the traditional method and other DL methods, CloudNet had higher accuracy in cloud and haze classification, as well as better performance in cirrus cloud recognition. CloudNet will be incorporated into the Open Access Satellite Image Service to facilitate change detection by using Sentinel-2 imagery on a regular and automatic basis.

Automatic in-line process quality control plays a crucial role to enhance production efficiency in the injection molding industry. Industry 4.0 is leading the productivity and efficiency of companies to minimize scrap rates and strive for zero-defect production, especially in the injection molding industry. In this study, a fully automated closed-loop injection molding (IM) setup with a communication platform via OPC UA was built in compliance with Industry 4.0. The setup included fully automated inline measurements, in-line data analysis, and an AI control system to set the new machine parameters via the OPC UA communication protocol. The surface quality of the injection molded parts was rated using the ResNet-18 convolutional neural network, which was trained on data gathered by a heuristic approach. Further, eight different machine learning models for predicting the part quality (weight, surface quality, and dimensional properties) and for predicting sensor data were trained using data from a variety of production information sources, including in-mold sensors, injection molding machine (IMM) sensors, ambient sensors, and inline product quality measurements. These models are the backbone of the AI control system, which is a heuristic model predictive control (MPC) method. This method was applied to find new sets of machine parameters during production to control the specified part quality feature. The control system and predictive models were successfully tested for two groups of quality features: Geometry control and surface quality control. Control parameters were limited to injection speed and holding pressure. Moreover, the geometry control was repeated with mold temperature as an additional control parameter.