Interpretable Trajectory Prediction via Counterfactual Responsibility

Interpretable Trajectory Prediction for Autonomous Vehicles
via Counterfactual Responsibility

Having an agent predictor that works across different contexts and is interpretable is important for safe autonomy in human-populated environments. The concept of responsibility is inherent in human actions and is enforced by legal systems. Thus, this work first formulates responsibility by counterfactual reasoning and employs responsibility to develop a hierarchical framework that wraps an underlying trajectory predictor with an interpretable layer.

How does it work?

We develop an interpretability framework centered around the concept of responsibility for motion forecasting. First, we formalize responsibility by employing counterfactual reasoning: if the query vehicle had acted differently, would it have been more responsible with respect to neighboring target vehicles? This assessment serves as the basis for constructing a high-level predictor for evaluating the responsibility level of agents. Subsequently, the anticipated responsibility level plays a pivotal role in guiding a lower-level trajectory predictor, enabling us to generate predictions attuned to the concept of responsibility.

Counterfactual Responsibility

Safety Responsibility:

Courtesy Responsibility:

if the query vehicle had acted differently, would it have been possible to maintain greater separation?

The queried agent could have maintained larger safety margin (by executing other possible motions) with respect to the target agent.
⚠️ irresponsible in the safety metric

if the query vehicle hadn’t been there, would the predicted motions of other agents have changed significantly?

There is a large difference in the distributions of the target agent’s motion with/without the queried vehicle’s presence.
⚠️ irresponsible in the courtesy metric

Responsibility-based Interpretability Framework

High-Level: Latent Abstraction Through Responsibility

Simplifies sequences of responsibility metrics into distinct levels by a hidden Markov model to encode long-term dependency

Low-Level: Responsibility-Aware Trajectory Prediction

Clusters the traffic dataset by responsibility levels and augments it with artificial trajectories from the underlying trajectory predictor.
Learns a reward function for each cluster to separate the real and artificial trajectories by inverse reinforcement learning.
Fine-tunes the underlying trajectory predictor or selects the the most likely motion with responsibility-dependent reward functions.

Results

Case study: overtaking in NuScenes

During the overtake, the blue car is aggressive and moves very close to the red car, which leads to safety decrease. Also, the blue car influences the red car’s possible motions and is thus irresponsible in courtesy. After overtake, the blue car does not interact with the red car and is classified as responsible.

Responsibility-Aware Prediction

Responsibility-dependent reward functions capture different modes of human behavior. Thus, responsibility-aware selection outperforms heuristic reward, which captures human behavior on average only. Additionally, we can generate diverse motions.

@inproceedings{hsu2023responsibility,
title={Interpretable Trajectory Prediction for Autonomous Vehicles viaCounterfactual Responsibility},
author={Hsu, Kai-Chieh and Leung, Karen and Chen, Yuxiao and Fisac, Jaime Fern\`andez and Pavone, Marco},
booktitle={IEEE/RSJ Int.\ Conf.\ on Intelligent Robots \& Systems},
year={2023},
pages={5918-5925},
doi={10.1109/IROS55552.2023.10341712}
}