Explore the vast range of titles available.

As robots acquire increasingly sophisticated skills and see increasingly complex and varied environments, the threat of an edge case or anomalous failure is ever present. For example, Tesla cars have seen interesting failure modes ranging from autopilot disengagements due to inactive traffic lights carried by trucks to phantom braking caused by images of stop signs on roadside billboards. These system-level failures are not due to failures of any individual component of the autonomy stack but rather system-level deficiencies in semantic reasoning. Such edge cases, which we call semantic anomalies, are simple for a human to disentangle yet require insightful reasoning. To this end, we study the application of large language models (LLMs), endowed with broad contextual understanding and reasoning capabilities, to recognize such edge cases and introduce a monitoring framework for semantic anomaly detection in vision-based policies. Our experiments apply this framework to a finite state machine policy for autonomous driving and a learned policy for object manipulation. These experiments demonstrate that the LLM-based monitor can effectively identify semantic anomalies in a manner that shows agreement with human reasoning. Finally, we provide an extended discussion on the strengths and weaknesses of this approach and motivate a research outlook on how we can further use foundation models for semantic anomaly detection. Our project webpage can be found at https://sites.google.com/view/llm-anomaly-detection.

As the vital technology of natural language understanding, sentence representation reasoning technology mainly focuses on sentence representation methods and reasoning models. Although the performance has been improved, there are still some problems, such as incomplete sentence semantic expression, lack of depth of reasoning model, and lack of interpretability of the reasoning process. Given the reasoning model’s lack of reasoning depth and interpretability, a deep fusion matching network is designed in this paper, which mainly includes a coding layer, matching layer, dependency convolution layer, information aggregation layer, and inference prediction layer. Based on a deep matching network, the matching layer is improved. Furthermore, the heuristic matching algorithm replaces the bidirectional long-short memory neural network to simplify the interactive fusion. As a result, it improves the reasoning depth and reduces the complexity of the model; the dependency convolution layer uses the tree-type convolution network to extract the sentence structure information along with the sentence dependency tree structure, which improves the interpretability of the reasoning process. Finally, the performance of the model is verified on several datasets. The results show that the reasoning effect of the model is better than that of the shallow reasoning model, and the accuracy rate on the SNLI test set reaches 89.0%. At the same time, the semantic correlation analysis results show that the dependency convolution layer is beneficial in improving the interpretability of the reasoning process.

Web of Things (WoT) facilitates the discovery and interoperability of Internet of Things (IoT) devices in a cyber-physical system (CPS). Moreover, a uniform knowledge representation of physical resources is quite necessary for further composition, collaboration, and decision-making process in CPS. Though several efforts have integrated semantics with WoT, such as knowledge engineering methods based on semantic sensor networks (SSN), it still could not represent the complex relationships between devices when dynamic composition and collaboration occur, and it totally depends on manual construction of a knowledge base with low scalability. In this paper, to addresses these limitations, we propose the semantic Web of Things (SWoT) framework for CPS (SWoT4CPS). SWoT4CPS provides a hybrid solution with both ontological engineering methods by extending SSN and machine learning methods based on an entity linking (EL) model. To testify to the feasibility and performance, we demonstrate the framework by implementing a temperature anomaly diagnosis and automatic control use case in a building automation system. Evaluation results on the EL method show that linking domain knowledge to DBpedia has a relative high accuracy and the time complexity is at a tolerant level. Advantages and disadvantages of SWoT4CPS with future work are also discussed.

Robots operating in everyday environments need to effectively perceive, model, and infer semantic properties of objects. Existing knowledge reasoning frameworks only model binary relations between an object’s class label and its semantic properties, unable to collectively reason about object properties detected by different perception algorithms and grounded in diverse sensory modalities. We bridge the gap between multimodal perception and knowledge reasoning by introducing an n-ary representation that models complex, inter-related object properties. To tackle the problem of collecting n-ary semantic knowledge at scale, we propose transformer neural networks that generalize knowledge from observations of object instances by learning to predict single missing properties or predict joint probabilities of all properties. The learned models can reason at different levels of abstraction, effectively predicting unknown properties of objects in different environmental contexts given different amounts of observed information. We quantitatively validate our approach against prior methods on LINK, a unique dataset we contribute that contains 1457 object instances in different situations, amounting to 15 multimodal properties types and 200 total properties. Compared to the top-performing baseline, a Markov Logic Network, our models obtain a 10% improvement in predicting unknown properties of novel object instances while reducing training and inference time by more than 150 times. Additionally, we apply our work to a mobile manipulation robot, demonstrating its ability to leverage n-ary reasoning to retrieve objects and actively detect object properties. The code and data are available at https://github.com/wliu88/LINK.

Withthe widespread deployment of intelligent terminals, mobile payment platforms, and Internet of Things devices, security systems are being progressively transformed from traditional transaction outcome analysis toward an intelligent perception paradigm centered on user behavior, device states, and environmental context. To address the challenges of multimodal data heterogeneity, non-independent and identically distributed data across nodes, and the difficulty of centralized modeling under privacy constraints in distributed scenarios, an artificial intelligence-driven federated multimodal security perception framework, namely FMS-LLM, is proposed. At its core, the framework introduces a Non-IID adaptive federated fusion mechanism that achieves dual-level alignment—structural alignment via parameter-level masks and semantic alignment via feature consistency constraints—to effectively mitigate cross-node distribution discrepancies. Additionally, an LLM-driven semantic enhancement module is developed, utilizing trend-guided token selection and inertia-suppression to map low-level sensing features into high-level risk semantic representations, thereby supporting logical reasoning and explainable decision-making. This framework takes user behavioral sensing data, device state information, environmental context data, and transaction behavior data as inputs, and constructs an integrated security analysis pipeline of “perception–collaboration–reasoning”. Experimental results on the distributed multimodal security perception task demonstrate that the proposed method achieves an Accuracy of 91.62%, a Precision of 91.04%, a Recall of 90.37%, an F1-score of 90.70%, and a ROC-AUC of 94.73%, consistently outperforming baseline methods including Logistic Regression, Random Forest, LSTM, the centralized multimodal deep model, FedAvg, FedProx, and MOON. Under strongly Non-IID conditions, when α=0.1, the model still maintains an Accuracy of 88.47% and an F1-score of 87.11%, demonstrating stronger cross-node robustness. The ablation study further indicates that the complete model attains the best classification performance while reducing communication cost to 18.92 MB/Round. These results demonstrate that the proposed method can effectively fuse multi-source sensing information under privacy-preserving conditions and support intelligent security perception tasks with higher accuracy, stronger robustness, and improved interpretability.

Smart cities provide integrated management and operation of urban data emerging within a city, supplying the infrastructure for smart city services and resolving various urban challenges. Nevertheless, cities continue to grapple with substantial issues, such as contagious diseases and terrorism, that pose severe financial and human risks. These problems sporadically arise in various locales, and current smart city frameworks lack the capability to autonomously identify and address these issues. The challenge intensifies especially when trying to recognize and respond to unprecedented problems. The primary objective of this research is to predict potential urban issues and support their resolution proactively. To achieve this, our system makes use of semantic reasoning to understand the ongoing situations within the city. In this process, the 5W1H principles serve as inference rules, guiding the extraction and consolidation of context. Firstly, utilizing domain-specific annotation templates, we craft a semantic graph by amalgamating information from various sources available in the city, such as municipal public data and IoT platforms. Subsequently, the system autonomously infers and accumulates contexts of situations occurring in the city using 5W1H-based reasoning. As a result, the accumulated contexts allow for inferring potential urban problems by identifying repeated disruptions in city services at specific times or locations and establishing connections among them. The main contribution of this paper lies in proposing a comprehensive conceptual model for the suggested system and presenting actual implementation cases and applicable use cases. These contributions facilitate awareness among city administrators and citizens within a smart city regarding potential problem-prone areas or times, thereby aiding in the preemptive identification and mitigation of urban challenges.

Building management systems (BMSs) are being implemented broadly by industries in recent decades. However, BMSs focus on specific domains, and when installed on the same building, they lack interoperability to work on a centralized user interface. On the other hand, BMSs interoperability allows the implementation of complex rules based on multi-domain contexts. The Building’s Reasoning for Intelligent Control Knowledge-based System (BRICKS) is a context-aware semantic rule-based system for the intelligent management of buildings’ energy and security. It uses ontologies and semantic web technologies to interact with different domains, taking advantage of cross-domain knowledge to apply context-based rules. This work upgrades the previously presented version of BRICKS by including services for energy consumption and generation forecast, demand response, a configuration user interface (UI), and a dynamic building monitoring and management UI. The case study demonstrates BRICKS deployed at different aggregation levels in the authors’ laboratory building, managing a demand response event and interacting autonomously with other BRICKS instances. The results validate the correct functioning of the proposed tool, which contributes to the flexibility, efficiency, and security of building energy systems.

In recent years, cross-modal retrieval has garnered significant attention due to its potential to bridge heterogeneous data modalities, particularly in aligning visual content with natural language. Despite notable progress, existing methods often struggle to accurately capture user intent when queries are expressed through complex or evolving instructions. To address this challenge, we propose a novel cross-modal representation learning framework that incorporates an instruction-aware dynamic query generation mechanism, augmented by the semantic reasoning capabilities of large language models (LLMs). The framework dynamically constructs and iteratively refines query representations conditioned on natural language instructions and guided by user feedback, thereby enabling the system to effectively infer and adapt to implicit retrieval intent. Extensive experiments on standard multimodal retrieval benchmarks demonstrate that our method significantly improves retrieval accuracy and adaptability, outperforming fixed-query baselines and showing enhanced cross-modal alignment and generalization across diverse retrieval tasks.

Considering that the largest part of end-use energy consumption worldwide is associated with the buildings sector, there is an inherent need for the conceptualization, specification, implementation, and instantiation of novel solutions in smart buildings, able to achieve significant reductions in energy consumption through the adoption of energy efficient techniques and the active engagement of the occupants. Towards the design of such solutions, the identification of the main energy consuming factors, trends, and patterns, along with the appropriate modeling and understanding of the occupants’ behavior and the potential for the adoption of environmentally-friendly lifestyle changes have to be realized. In the current article, an innovative energy-aware information technology (IT) ecosystem is presented, aiming to support the design and development of novel personalized energy management and awareness services that can lead to occupants’ behavioral change towards actions that can have a positive impact on energy efficiency. Novel information and communication technologies (ICT) are exploited towards this direction, related mainly to the evolution of the Internet of Things (IoT), data modeling, management and fusion, big data analytics, and personalized recommendation mechanisms. The combination of such technologies has resulted in an open and extensible architectural approach able to exploit in a homogeneous, efficient and scalable way the vast amount of energy, environmental, and behavioral data collected in energy efficiency campaigns and lead to the design of energy management and awareness services targeted to the occupants’ lifestyles. The overall layered architectural approach is detailed, including design and instantiation aspects based on the selection of set of available technologies and tools. Initial results from the usage of the proposed energy aware IT ecosystem in a pilot site at the University of Murcia are presented along with a set of identified open issues for future research.