Explore the vast range of titles available.

The critical task of recognizing human–object interactions (HOI) finds its application in the domains of surveillance, security, healthcare, assisted living, rehabilitation, sports, and online learning. This has led to the development of various HOI recognition systems in the recent past. Thus, the purpose of this study is to develop a novel graph-based solution for this purpose. In particular, the proposed system takes sequential data as input and recognizes the HOI interaction being performed in it. That is, first of all, the system pre-processes the input data by adjusting the contrast and smoothing the incoming image frames. Then, it locates the human and object through image segmentation. Based on this, 12 key body parts are identified from the extracted human silhouette through a graph-based image skeletonization technique called image foresting transform (IFT). Then, three types of features are extracted: full-body feature, point-based features, and scene features. The next step involves optimizing the different features using isometric mapping (ISOMAP). Lastly, the optimized feature vector is fed to a graph convolution network (GCN) which performs the HOI classification. The performance of the proposed system was validated using three benchmark datasets, namely, Olympic Sports, MSR Daily Activity 3D, and D3D-HOI. The results showed that this model outperforms the existing state-of-the-art models by achieving a mean accuracy of 94.1% with the Olympic Sports, 93.2% with the MSR Daily Activity 3D, and 89.6% with the D3D-HOI datasets.

This paper introduces a video representation based on dense trajectories and motion boundary descriptors. Trajectories capture the local motion information of the video. A dense representation guarantees a good coverage of foreground motion as well as of the surrounding context. A state-of-the-art optical flow algorithm enables a robust and efficient extraction of dense trajectories. As descriptors we extract features aligned with the trajectories to characterize shape (point coordinates), appearance (histograms of oriented gradients) and motion (histograms of optical flow). Additionally, we introduce a descriptor based on motion boundary histograms (MBH) which rely on differential optical flow. The MBH descriptor shows to consistently outperform other state-of-the-art descriptors, in particular on real-world videos that contain a significant amount of camera motion. We evaluate our video representation in the context of action classification on nine datasets, namely KTH, YouTube, Hollywood2, UCF sports, IXMAS, UIUC, Olympic Sports, UCF50 and HMDB51. On all datasets our approach outperforms current state-of-the-art results.

Recognising human actions in video is a challenging task in real-world. Dense trajectory (DT) offers accurate recording of motions over time that is rich in dynamic information. However, DT models lack the mechanism to distinguish dominant motions from secondary ones over separable frequency bands and directions. By contrast, deep learning-based methods are promising over the challenge though still suffering from limited capacity in handling complex temporal information, not mentioning huge datasets needed to guide the training. To take the advantage of semantical meaningful and “handcrafted” video features through feature engineering, this study integrates the discrete wavelet transform (DWT) technique into the DT model for gaining more descriptive human action features. Through exploring the pre-trained dual-stream CNN-RNN models, learned features can be integrated with the handcrafted ones to satisfy stringent analytical requirements within the spatial-temporal domain. This hybrid feature framework generates efficient Fisher Vectors through a novel Bag of Temporal Features scheme and is capable of encoding video events whilst speeding up action recognition for real-world applications. Evaluation of the design has shown superior recognition performance over existing benchmark systems. It has also demonstrated promising applicability and extensibility for solving challenging real-world human action recognition problems.

The Dense Trajectories concept is one of the most successful approaches in action recognition, suitable for scenarios involving a significant amount of motion. However, due to noise and background motion, many generated trajectories are irrelevant to the actual human activity and can potentially lead to performance degradation. In this paper, we propose Localized Trajectories as an improved version of Dense Trajectories where motion trajectories are clustered around human body joints provided by RGB-D cameras and then encoded by local Bag-of-Words. As a result, the Localized Trajectories concept provides an advanced discriminative representation of actions. Moreover, we generalize Localized Trajectories to 3D by using the depth modality. One of the main advantages of 3D Localized Trajectories is that they describe radial displacements that are perpendicular to the image plane. Extensive experiments and analysis were carried out on five different datasets.

Dense trajectory has become one of the most successful hand-crafted features for action recognition. However, most of the existing dense trajectories based methods ignore the relationship between trajectories. In this paper, we propose multiple relative descriptors of trajectories to model the relative information of pairs of trajectories. Specifically, we present relative motion descriptors and relative location descriptors, which are utilized to capture the relative motion information and relative location information respectively. Moreover, we present relative deep feature descriptors which combine the deep features with hand-crafted features. By aggregating the above descriptors, we obtain the fixed-length representation regardless of the various duration of input video. The experimental results on three standard datasets demonstrate the superiority of our method.

Gait is one of the most important biomarkers for Parkinson’s disease (PD). Nonetheless, current clinical diagnosis to quantify locomotion patterns uses coarse approximations, from a set of reduced marker-based trajectories. This approximation, among others, results restrictive, invasive, alters natural gait gestures, and leaves out relevant PD patterns. This paper introduces a new computational approach to quantify, classify and explain Parkinson gait patterns using a markerless video strategy. The core of the work is a local volumetric covariance to codify motion patterns during locomotion. Such covariance codifies convolutional pre-trained features tracked along a set of dense trajectories which represent subject’s gait. Covariance pattern computation involves an integral strategy to remain efficient in terms of computational cost. The proposed method was evaluated on 176 gait video sequences of a total of 22 patients among control and diagnosed with PD. The proposed approach achieved a remarkable average accuracy of 96.59% (± 0.13) with a sensitivity of 98.86%, specificity of 94.31%, and precision of 94.56%. These results suggest that the proposed approach may support clinical PD diagnosis and analysis using ordinary videos.

In the field of computer visions, it is demanding and challenging to determine fabric categories in accordance with appearance changes and multi-frame motion information from a video. Investigating recent impressive results on textile fabric classification techniques, we observed that motion-based video analytics were overlooked in the prior studies. To address this technological gap, a framework called Two-Stream+ , which employs deep neural networks to classify textile fabrics through small motions in videos is proposed. At the heart of the Two-Stream+ framework, the motion information of textile (e.g., flow trajectories and dense trajectories) was used to expose the material properties. More specifically, we advocate for fusing spatial and temporal Convolutional Neural Networks (i.e., ConvNets ) towers at the first fully connected layer. In addition, deformable convolution is used in Residual Networks (i.e., ResNet ) to enhance the transformation modeling capability of ConvNets. Testing a publicly available database, a conducted experiments is used to illustrate that the Two-Stream+ architecture has distinct advantages over the state-of-the-art architectures for classifying textile fabrics.

A new method for abnormal behavior detection is proposed using deep learning. Two SDAEs are utilized to automatically learn appearance feature and motion feature respectively, which are constrained in the space–time volume along dense trajectories that carry rich motion information to reduce the computational complexity. The vision words are exploited to describe behavior by the bag of words, and in order to reduce feature dimensions, the Agglomerative Information Bottleneck approach is used for vocabulary compression. An adaptive feature fusion method is adopted to enhance the discriminating power of these features. A sparse representation is exploited to abnormal behavior detection, which improve the detection accuracy. The proposed method is verified on the public dataset BEHAVE and BOSS and the results indicate the effectiveness of the proposed method.

We present a vision-inertial system which combines two RGB-Depth devices together with a wearable inertial movement unit in order to detect activities of the daily living. From multi-view videos, we extract dense trajectories enriched with a histogram of normals description computed from the depth cue and bag them into multi-view codebooks. During the later classification step a multi-class support vector machine with a RBF- X 2 kernel combines the descriptions at kernel level. In order to perform action detection from the videos, a sliding window approach is utilized. On the other hand, we extract accelerations, rotation angles, and jerk features from the inertial data collected by the wearable placed on the user’s dominant wrist. During gesture spotting, a dynamic time warping is applied and the aligning costs to a set of pre-selected gesture sub-classes are thresholded to determine possible detections. The outputs of the two modules are combined in a late-fusion fashion. The system is validated in a real-case scenario with elderly from an elder home. Learning-based fusion results improve the ones from the single modalities, demonstrating the success of such multimodal approach.

Purpose Diagnosis and treatment of Parkinson’s Disease (PD) are typically supported by a kinematic gait analysis. Nonetheless, the main drawbacks of the classical analysis, based on a reduced set of markers, are the loss of small dynamical changes, the invasive methodology, and the sparse representation from few points, restricting the disease analysis. This work aims to perform a robust regional kinematic characterization, which may result in a potential digital biomarker of the disease to complement personalized analysis, treatment and monitoring of PD. Methods This work introduces a markerless computational framework based on a full body-segment kinematic characterization related with PD motor alterations. Firstly, a set of dense motion trajectories are computed to represent locomotion. Such trajectories are grouped using a deep learning based body segmentation, that partitions the human silhouette into regions corresponding to the head, trunk and limbs. Each resultant region is described using dartboard -like kinematic histograms computed along the trajectories. Results The proposed approach was validated using different pretrained classification models. The proposed method was evaluated on a set of 11 control subjects and 11 PD patients, achieving an average accuracy of 99.62 % for lower-limbs and head regions. Conclusion This work proved to be effective to classify Parkinsonian patterns w.r.t control gaits. A major contribution of the proposed strategy is the capability to recover kinematic patterns in different body segments, particularly, for head and trunk regions, which turned out to be a decisive PD biomarker.