Explore the vast range of titles available.

The incredible advancements in artificial intelligence over the past decade have enabled technologies that once lived in research labs to now interact with users from all walks of life. As these agents evolve digitally and expand their physical presence through robotics, the risks associated with human interaction grow—necessitating stronger assurances. These risks stem from the inherent difficulty of deploying machine learning models, which must sense and interpret dynamic environments and human behavior, compared to more predictable, classical software systems.This thesis explores how deep learning can enhance human-robot interaction (HRI) by enabling general, flexible representations that support robust and unconstrained language grounding. Through the development of a neural object representation system, I demonstrate improved performance over prior category-based methods on a challenging, crowd-sourced dataset. Building on this, I introduce joint language-vision modeling, which further enhances generalization and usability, and extends the system to operate directly on speech—broadening accessibility for diverse user populations.However, the generalization power of deep learning introduces new challenges, especially in safety-critical scenarios involving physically embodied robots. To address this, I propose a data-centric threat model for adversarial attacks on vision systems, exposing the limitations of existing defenses. Extending this analysis to human-sensing systems, I identify disparities in adversarial robustness, particularly for users with diverse speech characteristics. Through a comprehensive case study, I show that while robustness training often entails performance trade-offs, rejection-based defenses—augmented through sampling—can achieve a better balance between robustness, performance, and equity.Finally, I revisit concept-based learning through the lens of assurance, introducing end-to-end differentiable neurosymbolic reasoning to align neural perception with symbolic tasks in both vision and speech. These methods improve interpretability, robustness, and fairness, while enabling alignment verification. Collectively, this work reflects a broader methodology: advancing capabilities, quantifying emerging risks, and designing mitigations that inform new paradigms for assured AI. This cycle—of innovation, analysis, and refinement—serves as a foundation for developing safe, equitable, and assured AI systems.

Over the past decade, the machine learning security community has developed a myriad of defenses for evasion attacks. An understudied question in that community is: for whom do these defenses defend? This work considers common approaches to defending learned systems and how security defenses result in performance inequities across different sub-populations. We outline appropriate parity metrics for analysis and begin to answer this question through empirical results of the fairness implications of machine learning security methods. We find that many methods that have been proposed can cause direct harm, like false rejection and unequal benefits from robustness training. The framework we propose for measuring defense equality can be applied to robustly trained models, preprocessing-based defenses, and rejection methods. We identify a set of datasets with a user-centered application and a reasonable computational cost suitable for case studies in measuring the equality of defenses. In our case study of speech command recognition, we show how such adversarial training and augmentation have non-equal but complex protections for social subgroups across gender, accent, and age in relation to user coverage. We present a comparison of equality between two rejection-based defenses: randomized smoothing and neural rejection, finding randomized smoothing more equitable due to the sampling mechanism for minority groups. This represents the first work examining the disparity in the adversarial robustness in the speech domain and the fairness evaluation of rejection-based defenses.

The ability to transfer adversarial attacks from one model (the surrogate) to another model (the victim) has been an issue of concern within the machine learning (ML) community. The ability to successfully evade unseen models represents an uncomfortable level of ease toward implementing attacks. In this work we note that as studied, current transfer attack research has an unrealistic advantage for the attacker: the attacker has the exact same training data as the victim. We present the first study of transferring adversarial attacks focusing on the data available to attacker and victim under imperfect settings without querying the victim, where there is some variable level of overlap in the exact data used or in the classes learned by each model. This threat model is relevant to applications in medicine, malware, and others. Under this new threat model attack success rate is not correlated with data or class overlap in the way one would expect, and varies with dataset. This makes it difficult for attacker and defender to reason about each other and contributes to the broader study of model robustness and security. We remedy this by developing a masked version of Projected Gradient Descent that simulates class disparity, which enables the attacker to reliably estimate a lower-bound on their attack's success.

To create usable and deployable Artificial Intelligence (AI) systems, there requires a level of assurance in performance under many different conditions. Many times, deployed machine learning systems will require more classic logic and reasoning performed through neurosymbolic programs jointly with artificial neural network sensing. While many prior works have examined the assurance of a single component of the system solely with either the neural network alone or entire enterprise systems, very few works have examined the assurance of integrated neurosymbolic systems. Within this work, we assess the assurance of end-to-end fully differentiable neurosymbolic systems that are an emerging method to create data-efficient and more interpretable models. We perform this investigation using Scallop, an end-to-end neurosymbolic library, across classification and reasoning tasks in both the image and audio domains. We assess assurance across adversarial robustness, calibration, user performance parity, and interpretability of solutions for catching misaligned solutions. We find end-to-end neurosymbolic methods present unique opportunities for assurance beyond their data efficiency through our empirical results but not across the board. We find that this class of neurosymbolic models has higher assurance in cases where arithmetic operations are defined and where there is high dimensionality to the input space, where fully neural counterparts struggle to learn robust reasoning operations. We identify the relationship between neurosymbolic models' interpretability to catch shortcuts that later result in increased adversarial vulnerability despite performance parity. Finally, we find that the promise of data efficiency is typically only in the case of class imbalanced reasoning problems.

The quantification of uncertainty is important for the adoption of machine learning, especially to reject out-of-distribution (OOD) data back to human experts for review. Yet progress has been slow, as a balance must be struck between computational efficiency and the quality of uncertainty estimates. For this reason many use deep ensembles of neural networks or Monte Carlo dropout for reasonable uncertainty estimates at relatively minimal compute and memory. Surprisingly, when we focus on the real-world applicable constraint of \\( 1\\%\\) false positive rate (FPR), prior methods fail to reliably detect OOD samples as such. Notably, even Gaussian random noise fails to trigger these popular OOD techniques. We help to alleviate this problem by devising a simple adversarial training scheme that incorporates an attack of the epistemic uncertainty predicted by the dropout ensemble. We demonstrate this method improves OOD detection performance on standard data (i.e., not adversarially crafted), and improves the standardized partial AUC from near-random guessing performance to \\( 0.75\\).

Non-negative matrix factorization (NMF) with missing-value completion is a well-known effective Collaborative Filtering (CF) method used to provide personalized user recommendations. However, traditional CF relies on the privacy-invasive collection of users' explicit and implicit feedback to build a central recommender model. One-shot federated learning has recently emerged as a method to mitigate the privacy problem while addressing the traditional communication bottleneck of federated learning. In this paper, we present the first unsupervised one-shot federated CF implementation, named FedSPLIT, based on NMF joint factorization. In our solution, the clients first apply local CF in-parallel to build distinct client-specific recommenders. Then, the privacy-preserving local item patterns and biases from each client are shared with the processor to perform joint factorization in order to extract the global item patterns. Extracted patterns are then aggregated to each client to build the local models via knowledge distillation. In our experiments, we demonstrate the feasibility of our approach with standard recommendation datasets. FedSPLIT can obtain similar results than the state of the art (and even outperform it in certain situations) with a substantial decrease in the number of communications.

Learning to understand grounded language, which connects natural language to percepts, is a critical research area. Prior work in grounded language acquisition has focused primarily on textual inputs. In this work we demonstrate the feasibility of performing grounded language acquisition on paired visual percepts and raw speech inputs. This will allow interactions in which language about novel tasks and environments is learned from end users, reducing dependence on textual inputs and potentially mitigating the effects of demographic bias found in widely available speech recognition systems. We leverage recent work in self-supervised speech representation models and show that learned representations of speech can make language grounding systems more inclusive towards specific groups while maintaining or even increasing general performance.

We propose a cross-modality manifold alignment procedure that leverages triplet loss to jointly learn consistent, multi-modal embeddings of language-based concepts of real-world items. Our approach learns these embeddings by sampling triples of anchor, positive, and negative data points from RGB-depth images and their natural language descriptions. We show that our approach can benefit from, but does not require, post-processing steps such as Procrustes analysis, in contrast to some of our baselines which require it for reasonable performance. We demonstrate the effectiveness of our approach on two datasets commonly used to develop robotic-based grounded language learning systems, where our approach outperforms four baselines, including a state-of-the-art approach, across five evaluation metrics.

We present an overview of the National Science Foundation’s Daniel K. Inouye Solar Telescope (DKIST), its instruments, and support facilities. The 4 m aperture DKIST provides the highest-resolution observations of the Sun ever achieved. The large aperture of DKIST combined with state-of-the-art instrumentation provide the sensitivity to measure the vector magnetic field in the chromosphere and in the faint corona, i.e. for the first time with DKIST we will be able to measure and study the most important free-energy source in the outer solar atmosphere – the coronal magnetic field. Over its operational lifetime DKIST will advance our knowledge of fundamental astronomical processes, including highly dynamic solar eruptions that are at the source of space-weather events that impact our technological society. Design and construction of DKIST took over two decades. DKIST implements a fast (f/2), off-axis Gregorian optical design. The maximum available field-of-view is 5 arcmin. A complex thermal-control system was implemented in order to remove at prime focus the majority of the 13 kW collected by the primary mirror and to keep optical surfaces and structures at ambient temperature, thus avoiding self-induced local seeing. A high-order adaptive-optics system with 1600 actuators corrects atmospheric seeing enabling diffraction limited imaging and spectroscopy. Five instruments, four of which are polarimeters, provide powerful diagnostic capability over a broad wavelength range covering the visible, near-infrared, and mid-infrared spectrum. New polarization-calibration strategies were developed to achieve the stringent polarization accuracy requirement of 5×10 −4 . Instruments can be combined and operated simultaneously in order to obtain a maximum of observational information. Observing time on DKIST is allocated through an open, merit-based proposal process. DKIST will be operated primarily in “service mode” and is expected to on average produce 3 PB of raw data per year. A newly developed data center located at the NSO Headquarters in Boulder will initially serve fully calibrated data to the international users community. Higher-level data products, such as physical parameters obtained from inversions of spectro-polarimetric data will be added as resources allow.

N2O remains a major greenhouse gas and contributor to global warming, therefore developing a catalyst that can decompose N2O at low temperatures is of global importance. We have investigated the use of LaSrCoFeOx perovskites for N2O decomposition and the effect of surface area, A and B site elements, Co–O bond strength, redox capabilities and oxygen mobility have been studied. It was found that by using a citric acid preparation method, perovskites with strong redox capabilities and weak Co–O bonds can be formed at relatively low calcination temperatures (550 °C) resulting in highly active catalysts. The enhanced activity is related to the presence of highly mobile oxygen species. Oxygen recombination on the catalyst surface is understood to be a prominent rate limiting step for N2O decomposition. Here the reduced strength of Co–O bonds and mobile lattice oxygen species suggest that the surface oxygen species have enhanced mobility, aiding recombination, and subsequent regeneration of the active sites. La0.75Sr0.25Co0.81Fe0.19Ox prepared by citric acid method converted 50% of the N2O in the feed (T50) at 448 °C.Graphic Abstract