Workshop on Public Trust in Autonomous Systems
Full-day workshop at the 2025 IEEE International Conference on Robotics & Automation in Atlanta, GA, USA
Accepted Papers
The SET Perceptual Factors Framework: Towards Assured Perception for Autonomous Systems
Author: Troi Williams
Abstract: Future autonomous systems promise significant societal benefits, yet their deployment raises concerns about safety and trustworthiness. A key concern is assuring the reliability of robot perception, as perception seeds safe decision-making. Failures in perception are often due to complex yet common environmental factors and can lead to accidents that erode public trust. To address this concern, we introduce the SET (Self, Environment, and Target) Perceptual Factors Framework. We designed the framework to systematically analyze how factors such as weather, occlusion, or sensor limitations negatively impact perception. To achieve this, the framework employs SET State Trees to categorize where such factors originate and SET Factor Trees to model how these sources and factors impact perceptual tasks like object detection or pose estimation. Next, we develop Perceptual Factor Models using both trees to quantify the uncertainty for a given task. Our framework aims to promote rigorous safety assurances and cultivate greater public understanding and trust in autonomous systems by offering a transparent and standardized method for identifying, modeling, and communicating perceptual risks.
Uncertainty-aware planning using deep ensembles and constrained trajectory optimization for social navigation
Authors: Anshul Nayak, Azim Eskandarian
Abstract: Human motion is stochastic and ensuring safe robot navigation in a pedestrian-rich environment requires proactive decision-making. Past research relied on incorporating deterministic future states of surrounding pedestrians which can be overconfident leading to unsafe robot behavior. The current paper proposes a predictive uncertainty-aware planner that integrates neural network based probabilistic trajectory prediction into planning. Our method uses a deep ensemble based network for probabilistic forecasting of surrounding humans and integrates the predictive uncertainty as constraints into the planner. We compare numerous constraint satisfaction methods on the planner and evaluated its performance on real world pedestrian datasets. Further, offline robot navigation was carried out on out-of-distribution pedestrian trajectories inside a narrow corridor.
Explanations for Object Avoidance with LLMs
Authors: Courtney Young, Sehoon Ha, J. Taery Kim
Abstract: As robots increasingly navigate dynamic human environments, transparent decision-making becomes essential for building user trust. This paper presents a modular explanation system that interprets object avoidance behavior in mobile robots, using a large language model (LLM). Our system combines object detection and depth data to prompt an LLM to produce natural language explanations, without modifying the underlying navigation policy. We evaluate the system in indoor scenarios using a TurtleBot4 platform and assess the perceived quality and trustworthiness of the explanations through a user study. Results suggest that the quality of LLM-generated explanations can enhance human understanding of autonomous robot behavior, improving trust in robot navigation.
Safety Standards and Risk Evaluation for Physical Human-Robot Interaction
Authors: Jennifer Molnar, Scott Lovald, Ian Campbell, Sarah Easley
Abstract: The existing robotic safety standards landscape has many origins in industrial robotics, where physical contact with a robot could be catastrophic. However, in modern collaborative robot domains, standards limiting physical contact forces to sub-pain levels or below thresholds for minor scrapes and bruises may not be relevant for all applications. Conducting robot or application-specific simulations or experiments can provide greater context to the actual risk presented by a particular robot or category of robots outside of universal standards. This abstract contributes an overview of evaluation methods relevant to common mechanical safety concerns.
Implicit Behavioral Cues for Enhancing Trust and Comfort in Robot Social Navigation
Authors: Yi Lian, J. Taery Kim, Sehoon Ha
Abstract: Robots navigating public spaces must move not only safely, but in ways that are intuitive and socially appropriate. This study investigates how implicit motion cues—such as slowing down or adjusting trajectory—affect pedestrian comfort, trust, and clarity of intent. Using a TurtleBot4 and a Wizard-of-Oz setup, we tested five behaviors: no cue, sudden stop, speed reduction, curved trajectory, and verbal announcement. In hallway encounters with 15 participants, we collected subjective ratings and video analysis of pedestrian behavior. Results show that trajectory-based cues significantly improve perceived comfort and trust, while abrupt or neutral behaviors lead to discomfort and hesitation. These findings highlight the importance of implicit motion cues for legible and socially aware robot navigation.
The Importance of Word Choice for Trusting Robot Explanations
Authors: Gregory LeMasurier, Giordano Arcieri, Holly A. Yanco
Abstract: It is especially important that robots are able to explain their failures in a manner that helps a person understand what went wrong and to appropriately align their trust with the system. In this work we updated our Generative explanation system to create explanations catered to novices. We also leverage context to help ground the language used in explanations with the environment. Through our study we highlight the importance of word choice on people's perception of a robot's trustworthiness.
Investigating Human Decision-Making in High-Stakes Scenarios: Implications for an Autonomous Decision-Making Aids in Human-Robot Teams
Authors: Nathan Uhunsere, Russell Perkins, Neil Shortland, Paul Robinette
Abstract: As autonomous systems become more integrated into high-stakes environments—such as military or healthcare settings—the need for robots to understand and adapt to a human partner's decision-making process is crucial. Establishing high trust in human-robot teaming ensures efficient collaboration. This paper presents an experiment designed to investigate factors influencing human decision-making in high-risk scenarios. The ultimate goal of this understanding is to develop autonomous systems capable of assisting humans in high-stakes decision-making—maximizing both comfort and task efficiency. Using Qualtrics and recruiting 100 military veterans via Prolific, participants navigated a four-step branching scenario involving ethical dilemmas between Tactical Evacuation methods. Following the scenario, participants completed the Schwartz Maximization Scale. Preliminary results showed maximization scores followed a Gaussian distribution, but initial analysis revealed no strong correlation between those scores and decision choices. Future work will employ Hidden Markov Models (HMM) and Multilevel Logistic Regression (MLR), with larger datasets and perceived risk metrics, to explore these relationships and inform the design of robotic systems capable of assisting with decision making.
Calibrating AI Trust in Complementary Human-AI Collaboration
Authors: Hanjiang Hu, Yifan Sun, Changliu Liu
Abstract: Human-AI collaboration is a powerful paradigm in decision-making systems, where humans and AI contribute different strengths with clear complementarity. Yet, achieving optimal team performance depends critically on proper trust in AI, ensuring humans rely on AI appropriately. In real-world scenarios, humans often lack the expertise or performance transparency to judge AI accuracy directly, creating a gap in appropriate trust calibration. In this paper, we address this challenge through three key contributions: (1) we propose a theoretical framework modeling the evolution of human trust in AI over time under AI performance uncertainty, (2) we investigate two self-calibrating trust methods, an instance-based cognitive model and a reinforcement learning (RL) model that learns trust calibration policies from experience, and (3) we conduct simulations comparing both approaches against a rule-based baseline under dynamically varying AI performance. Results show that RL-based trust calibration outperforms others in cumulative performance, while instance-based calibration offers interpretability and sample efficiency. These findings offer pathways for safe and adaptive trust alignment in human-AI collaboration toward trustworthy autonomy.
Explaining Robot Navigation with Semantic XAI Visualizations in Virtual Reality
Authors: Jorge de Heuvel, Sebastian Müller, Marlene Wessels, Aftab Akhtar, Christian Bauckhage, Maren Bennewitz
Abstract: End-to-end robot policies trained via reinforcement learning (RL) achieve high performance but their black-box nature limits human understanding and trust. We present a virtual reality (VR) interface visualizing explainable AI (XAI) outputs and robot lidar perception by highlighting objects based on attribution scores. A user study (N=24) evaluated four visualization conditions (XAI, lidar, both, none). Results show that semantic projection of attributions significantly improves users’ understanding and perceived predictability of the robot, emphasizing the importance of integrating XAI and sensor visualizations for transparent HRI.
Bridging Emotion and Trust: A Proposed Model for Understanding Distrust Through Physiology
Authors: Zahra Rezaei Khavas, Paul Robinette
Abstract: Estimating affective states using physiological measures has gained significant attention in recent years, thanks to advancements in artificial intelligence and the availability of high-quality sensors. Researchers have developed models to assess the arousal and valence of primary human emotions, providing valuable insights into emotional states. At the same time, ongoing research has been conducted on modeling human trust and distrust using physiological signals. While some models attempt to predict trust levels from physiological markers, there is still no clear framework that maps trust-related affective states.
In this proposed work, we explore whether moral distrust and performance distrust elicit distinct physiological responses. Our preliminary observations suggest that humans react differently to violations of performance expectations and violations of moral expectations, raising key research questions: Do these two types of distrust produce distinct physiological patterns? Can they be mapped within the arousal-valence framework of emotions? To address this, we propose a novel approach that conceptualizes trust-related affective states in terms of arousal and valence, similar to existing emotion models. This study aims to contribute to the understanding of human trust by developing a framework that connects distrust to fundamental emotional dimensions.
Certifying Robustness of Learning-Based Pose Estimation Methods
Authors: Xusheng Luo, Changliu Liu
Abstract: This work addresses the certification of the local robustness of vision-based two-stage 6D object pose estimation. The two-stage method for object pose estimation achieves superior accuracy over the single-stage approach by first employing deep neural network-driven keypoint regression and then applying a Perspective-n-Point (PnP) technique. Despite advancements, the certification of these methods' robustness, especially in safety-critical scenarios, remains scarce. This research aims to fill this gap with a focus on their local robustness on the system level—the capacity to maintain robust estimations amidst semantic input perturbations. The core idea is to transform the certification of local robustness into a process of neural network verification for classification tasks. We validate its effects through evaluations on realistic perturbations. To our knowledge, this is the first study to certify the robustness of large-scale, keypoint-based pose estimation given images in real-world scenarios.
DynaMem: Online Dynamic Spatio-Semantic Memory for Open World Mobile Manipulation
Authors: Peiqi Liu, Zhanqiu Guo, Mohit Warke, Soumith Chintala, Chris Paxton, Nur Muhammad Mahi Shafiullah, Lerrel Pinto
Abstract: Significant progress has been made in open-vocabulary mobile manipulation, where the goal is for a robot to perform tasks in any environment given a natural language description. However, most current systems assume a static environment, which limits the system’s applicability in real-world scenarios where environments frequently change due to human intervention or the robot’s own actions. In this work, we present DynaMem, a new approach to open-world mobile manipulation that uses a dynamic spatio-semantic memory to represent a robot’s environment. DynaMem constructs a 3D data structure to maintain a dynamic memory of point clouds, and answers open-vocabulary object localization queries using multimodal LLMs or open-vocabulary features generated by state-of-the-art vision-language models. Powered by DynaMem, our robots can explore novel environments, search for objects not found in memory, and continuously update the memory as objects move, appear, or disappear in the scene. We run extensive experiments on the Stretch SE3 robots in three real and nine offline scenes, and achieve an average pick-and-drop success rate of 70% on non-stationary objects, a 3X improvement over state-of-the-art static systems.
Best Presentation Award
Implicit Behavioral Cues for Enhancing Trust and Comfort in Robot Social Navigation
Authors: Yi Lian, J. Taery Kim, Sehoon Ha
Abstract: Robots navigating public spaces must move not only safely, but in ways that are intuitive and socially appropriate. This study investigates how implicit motion cues—such as slowing down or adjusting trajectory—affect pedestrian comfort, trust, and clarity of intent. Using a TurtleBot4 and a Wizard-of-Oz setup, we tested five behaviors: no cue, sudden stop, speed reduction, curved trajectory, and verbal announcement. In hallway encounters with 15 participants, we collected subjective ratings and video analysis of pedestrian behavior. Results show that trajectory-based cues significantly improve perceived comfort and trust, while abrupt or neutral behaviors lead to discomfort and hesitation. These findings highlight the importance of implicit motion cues for legible and socially aware robot navigation.
Best Poster Award
Explaining Robot Navigation with Semantic XAI Visualizations in Virtual Reality
Authors: Jorge de Heuvel, Sebastian Müller, Marlene Wessels, Aftab Akhtar, Christian Bauckhage, Maren Bennewitz
Abstract: End-to-end robot policies trained via reinforcement learning (RL) achieve high performance but their black-box nature limits human understanding and trust. We present a virtual reality (VR) interface visualizing explainable AI (XAI) outputs and robot lidar perception by highlighting objects based on attribution scores. A user study (N=24) evaluated four visualization conditions (XAI, lidar, both, none). Results show that semantic projection of attributions significantly improves users’ understanding and perceived predictability of the robot, emphasizing the importance of integrating XAI and sensor visualizations for transparent HRI.
