Explore the vast range of titles available.

Enhanced Vision System (EVS) offers a display advantage that conventional devices lack, enabling interface information to be overlaid on real-world imagery. However, information overload, especially in complex environments, can reduce the recognizability of important information and impair decision-making. This study investigates a dual color-coding strategy to optimize the recognizability of Primary Information (PI) and Secondary Information (SI) in Head-Up Display–Enhanced Vision System (HUD-EVS) against complex backgrounds. The results show that adjusting the hue, saturation, and lightness of SI affects the recognizability of both PI and SI. Specifically, certain saturation (20% or 80%) and lightness (60%) combinations should be avoided to ensure PI prominence and maintain sufficient recognizability for SI. These findings provide insights for designing color-coding strategies for EVS, enhancing the recognizability of information on mobile devices.

Animal action recognition has a wide range of applications. With the rise of visual-language pretraining models (VLMs), new possibilities have emerged for action recognition. However, while current VLMs perform well on human-centric videos, they still struggle with animal videos. This is primarily due to the lack of domain-specific knowledge during model training and more pronounced intra-class variations compared to humans. To address these issues, we introduce Animal-CLIP, a specialized and efficient animal action recognition framework built upon existing VLMs. To address the lack of domain-specific knowledge in animal actions, we leverage the extensive expertise of large language models (LLMs) to automatically generate external prompts, thereby expanding the semantic scope of labels and enhancing the model’s generalization capability. To effectively integrate external knowledge into the model, we propose a knowledge-enhanced internal prompt fine-tuning approach. We design a text feature refinement module to reduce potential recognition inconsistencies. Furthermore, to address the high intra-class variation in animal actions, a novel category-specific prompting method is introduced to generate adaptive prompts to optimize the alignment between text and video features, facilitating more precise partitioning of the action space. Experimental results demonstrate that our method outperforms six previous action recognition methods across three large-scale multi-species, multi-action datasets and exhibits strong generalization capability on unseen animals.

This paper discusses the method of expert assessment based on the probabilistic approach of the paired comparisons method. This mathematical apparatus is the basis for special software integrated into the universal stand for prototyping the cockpit, which allows for a systematic expert assessment at this stand according to pre-selected criteria with the control of the consistency of the answers. Using this method, the task of determining the feasibility of displaying layers of synthetic and enhanced vision on the flying frame is solved, but in general, the method allows evaluating any elements of the displays and controls.

The variety of crops, differences in climate, and the multiplicity of disease symptoms make early identification and evaluation of leaf diseases a challenging task. Although deep-learning methods have been created for this very reason, they often need adjustments to suit various regions and types of crops. This study presents a vision transformer-based deep learning model incorporating Explainable Artificial Intelligence (XAI) features to effectively detect and classify leaf diseases. The model is evaluated using the New Plant Disease Dataset, encompassing data from 14 plants with 38 leaf disease classifications. The primary objective is to enhance accuracy in leaf disease classification through the utilization of vision transformer learning. The proposed model, LEViT (Leaf Disease Identification and Classification using an Enhanced Vision Transformers Model), builds upon the ViT_B_32 model. It incorporates additional layers such as dense layers, dropout, and L2 regularization to improve performance and mitigate overfitting. Specialized preprocessing techniques tailored for transformers are employed to refine data quality. Incorporating Grad-CAM (Gradient-weighted Class Activation Mapping) enhances interpretability, enabling the identification of crucial regions influencing the model’s decisions within images. The model exhibits remarkable efficacy, boasting a training accuracy of 95.22%, validation accuracy of 96.19%, and test accuracy of 92.33%. These results underscore its robust ability to accurately identify leaf diseases across diverse crop types. Comparative analysis with state-of-the-art models in the field demonstrates LEViT’s superior performance, establishing it as a leading solution for leaf disease detection and classification. LEViT’s advanced architecture, coupled with fine-tuning, positions it as a highly promising tool for advancing crop management and yield improvement, thereby significantly contributing to the field of precision agriculture.

PurposeVision transformers (ViT) detectors excel in processing natural images. However, when processing remote sensing images (RSIs), ViT methods generally exhibit inferior accuracy compared to approaches based on convolutional neural networks (CNNs). Recently, researchers have proposed various structural optimization strategies to enhance the performance of ViT detectors, but the progress has been insignificant. We contend that the frequent scarcity of RSI samples is the primary cause of this problem, and model modifications alone cannot solve it.Design/methodology/approachTo address this, we introduce a faster RCNN-based approach, termed QAGA-Net, which significantly enhances the performance of ViT detectors in RSI recognition. Initially, we propose a novel quantitative augmentation learning (QAL) strategy to address the sparse data distribution in RSIs. This strategy is integrated as the QAL module, a plug-and-play component active exclusively during the model’s training phase. Subsequently, we enhanced the feature pyramid network (FPN) by introducing two efficient modules: a global attention (GA) module to model long-range feature dependencies and enhance multi-scale information fusion, and an efficient pooling (EP) module to optimize the model’s capability to understand both high and low frequency information. Importantly, QAGA-Net has a compact model size and achieves a balance between computational efficiency and accuracy.FindingsWe verified the performance of QAGA-Net by using two different efficient ViT models as the detector’s backbone. Extensive experiments on the NWPU-10 and DIOR20 datasets demonstrate that QAGA-Net achieves superior accuracy compared to 23 other ViT or CNN methods in the literature. Specifically, QAGA-Net shows an increase in mAP by 2.1% or 2.6% on the challenging DIOR20 dataset when compared to the top-ranked CNN or ViT detectors, respectively.Originality/valueThis paper highlights the impact of sparse data distribution on ViT detection performance. To address this, we introduce a fundamentally data-driven approach: the QAL module. Additionally, we introduced two efficient modules to enhance the performance of FPN. More importantly, our strategy has the potential to collaborate with other ViT detectors, as the proposed method does not require any structural modifications to the ViT backbone.

Detecting surface defects in metal manufacturing is critical, as they can greatly affect product quality and production efficiency. While vision transformers (ViTs) have advanced in surface defect detection, challenges remain, particularly due to high variability and sample imbalance in metal defects. To address these challenges, this paper proposes a vision transformer model that combines receptive-field attention convolution (RFAConv) and context broadcasting median (CBM), referred to as RFAConv-CBM-ViT. Firstly, this paper introduces RFAConv in the patch embedding stage, which adaptively adjusts the receptive field size, enhancing the model’s ability to capture global features and improving its capability to handle defects of varying scales. Its advantage lies in improving the model’s accuracy and convergence speed without introducing additional computational overhead, while also enhancing its robustness in addressing complex defects. Secondly, this paper design a method called CBM to enhance the performance of the ViT backbone. CBM utilizes median tokens within the multi-layer perceptron to reduce the density of the attention map. By aggregating local contextual information, CBM effectively suppresses noise and irrelevant features, leading to improved model accuracy without increasing computational cost. Experimental results demonstrate that the proposed RFAConv-CBM-ViT model achieves competitive performance across multiple public metal surface defect datasets. This method significantly enhances the performance of vision transformers in metal surface defect detection tasks, demonstrating its potential for applications in the metal manufacturing industry. Code is available at https://github.com/Vzoooong/RFAConv-CBM-ViT/tree/main .

Visual smoke recognition remains a substantial challenging task due to: (1) the large variations of smoke color, texture, brightness and shape caused by complex environment; (2) the difficulties in data collection and insufficient smoke datasets. The novel Transformer has attracted increasing interests in computer vision, but it still falls behind state-of-the-art convolutional neural networks when trained on limited datasets. To improve the visual feature representation of smoke image and address the problem of too few smoke datasets in real scenes, this paper proposes a new convolution-enhanced vision Transformer network (CViTNet) for smoke recognition by introducing desirable properties of convolutional neural network into vision Transformer. Instead of the straight tokenization in vision Transformer, we firstly revisit the merits of convolutional neural network and design convolutional token embedding by overlapping convolution operation with stride on the token feature maps, achieving feature resolution reduction and channel capacity expansion. We then partition vision Transformer into multiple stages by convolutional token embedding and construct a hierarchical structure to enhance feature representation and reduce computational complexity. CViTNet enjoys the advantages of both CNN and Transformer. Finally, we validate our approach by conducting extensive experiments, showing that CViTNet is establishing a new stage-of-the-art detection accuracy that exceeds 99.54% on average with 4.49 M learnable parameters and 346 M FLOPs.

The use of automated skin lesion classification is still a disadvantage, since there is a great visual similarity between benign and malignant lesions. The majority of deep learning methods utilize dermoscopic images only, without taking into account clinical metadata employed by dermatologists on a regular basis. The following paper proposes a vision-graph multimodal framework that links Image encoding to graph neural networks based on metadata representation through the fusion of learnable attention. The framework focuses on three limitations, which are underutilization of clinical context, absence of interpretability, and suboptimal incorporation of modalities. Gradient-weighted Class Activation Mapping++ (Grad-CAM++) is used to obtain dual explainability of visual attention, and SHapley Additive exPlanations (SHAP) to obtain feature importance. Examining the HAM10000 and Derm7pt datasets, statistically significant advances (p < 0.001) of 89.3% and 92.1% accuracy are obtained, which is 4.1% and 2.7% higher than baselines that can only use images. Focusing on weight analysis will provide metadata with 37.7% averaged variance with an error of 8.4%, which confirms the clinical importance of multimodal modeling. The study of ablation shows that graph-based metadata encoding is 1.4% better than standard multilayer perceptron encoding (p = 0.003).

Breast Cancer (BC) remains a critical global health challenge, accounting for millions of lives annually. Most breast cancer detection techniques are designed for binary classification. However, further classifying these types into their subtypes remains challenging due to cell coherence, variations in colour distribution, and intraclass similarity. Additionally, these techniques are limited to specific magnification levels of input images. As a result, the need to develop multiclass classification techniques independent of magnification levels has become increasingly demanding. This research introduces an innovative approach based on an enhanced vision transformer (ViT) for BC analysis that seamlessly handles binary and multiclass classification tasks in a magnification-independent manner. The proposed framework incorporates convolutional layers into the ViT model to simultaneously enhance its capability to capture local and global features. This innovative adjustment also empowers the model to capture complex patterns and subtle features by overcoming the issue of different magnification levels. Extensive experiments are conducted on the benchmark BreaKHis dataset to validate the effectiveness of the proposed framework. It achieves an accuracy of 89.43% for binary classification (benign vs. malignant) and 57.94% and 74.71% for multiclass benign and multiclass malignant subclass classification, respectively. The results confirm that the proposed framework surpasses the performance of the original ViT model and other state-of-the-art techniques. Highlights • Introduction of CEV-BCA, an innovative approach to Breast Cancer Analysis • Seamlessly performs binary and multiclass classification tasks independent of magnification levels • Enhancement of ViT model with convolutional layers to capture local and global features effectively • Demonstrates superior performance compared to state-of-the-art techniques, achieving high accuracy for both binary and multiclass classification tasks Graphical Abstract

Vision transformers (ViTs) have recently outperformed convolutional neural networks (CNNs) across a variety of deep learning tasks. In the field of orthopedic medicine, the thighbone serves as a critical support structure for the lower body, and a timely and accurate diagnosis of its fractures is important to preventing lifelong walking disabilities. Despite the successes of CNNs in the computer-aided diagnosis of thighbone fractures, the potential of ViTs in this realm remains unexplored. Consequently, we initially explored the direct application of off-the-shelf ViT models on thighbone fracture detection but found the results did not fully satisfy the requirement of radiologists. To address this gap, we propose a one-stage hybrid method that combines enhanced vision transformers with the CNN attention mechanisms, specifically for thighbone fracture detection. Our method improves a pyramid vision transformer architecture and employs overlapping patch embedding to preserve the local continuity in X-rays. For dynamic feature fusion across spatial and scale dimensions, we use a series of attention mechanisms consisting of two distinct types: scale-aware attention and spatial-aware attention. These mechanisms can integrate feature maps output from the neck structure, thereby improving the representation of thighbone fractures. We validate the proposed method using a meticulously curated dataset of 4000 thighbone X-rays, annotated by experienced radiologists. Ablation studies confirm the effectiveness of each modification in our proposed framework. Experimental results show that our method achieves an average precision ( AP ) of 53.7% and an A P 50 of 87.0%, thereby surpassing all previous state-of-the-art methods in thighbone fracture detection.