Publications

NaturalADV: An Exploratory Framework to Balance Adversarial Strength and Stealth in Autonomous Driving Environments

Meriel von Stein

Deep Neural Networks (DNNs) have become integral to various real-world autonomous mobile systems, from self-driving cars to food delivery robots. However, current adversarial attack techniques often focus on maximizing the attack strength at the cost of naturalness, leading to examples that are easily detected by humans or deviate significantly from… Read more

Download: [Poster] [Repo] [Openreview]

Improving Robustness of DNN-Dependent ADS: Transforming Input Distributions to Account for Inference and System State

Meriel von Stein

Autonomous Driving Systems (ADSs) are becoming more advanced and ubiquitous, enabled by increasingly sophisticated deep neural networks (DNNs). As ADSs’ autonomy levels rise, so does the cost and complexity of their failures. Often, these failures arise when these DNNs are less robust than expected. In studying these systems, I found… Read more

Download: [Paper] [Slides]

Automated Generation of Transformations to Mitigate Sensor Hardware Migration in ADS

Meriel von Stein, Hongning Wang, Sebastian Elbaum

Autonomous driving systems (ADSs) rely on massive amounts of sensed data to train their underlying deep neural networks (DNNs). Common sensor hardware migrations can render an existing DNN-dependent pipeline inadequate. This necessitates the development of bespoke transformations to adapt new sensor data to the old trained network, or the retraining… Read more

Download: [Paper] [Repo] [Video]

PhysCov: Physical Test Coverage for Autonomous Vehicles

Carl Hildebrandt, Meriel von Stein, Sebastian Elbaum

Adequately covering the behaviors of autonomous vehicles ($AV$) is fundamental in their validation. However, quantifying such coverage is challenging as the $AV$s’ behavior is influenced by its physical environment that is often large and highly complex. This work builds on the insights that data sensed by the $AV$ provides a… Read more

Download: [Paper] [Repo]

DeepManeuver: Adversarial Test Generation for Trajectory Manipulation of Autonomous Vehicles

Meriel von Stein, David Shriver, Sebastian Elbaum

Adversarial test generation techniques aim to produce input perturbations that cause a DNN to compute incorrect outputs. For autonomous vehicles driven by a DNN, however, the effect of such perturbations are attenuated by other parts of the system and are less effective as vehicle state evolves. In this work we… Read more

Download: [Paper] [Poster] [Repo] [Video]

Finding Property Violations through Network Falsification: Challenges, Adaptations and Lessons Learned from OpenPilot

Meriel von Stein, Sebastian Elbaum

Openpilot is an open source system to assist drivers by providing features like automated lane centering and adaptive cruise control. Like most systems for autonomous vehicles, Openpilot relies on a sophisticated deep neural network (DNN) to provide its functionality, one that is susceptible to safety property violations that can lead… Read more

Download: [Paper] [Repo]

Preparing Software Engineers to Develop Robot Systems

Carl Hildebrandt, Meriel von Stein, Trey Woodlief, Sebastian Elbaum

Most undergraduates are not equipped to manage the unique challenges in developing software for modern robots, despite rapid expansion of the field. We here introduce a course we have designed and delivered to better prepare students to develop software for robot systems. It emphasizes the distinctive challenges of software development… Read more

Download: [Paper] [Repo] [Video]

Automated Environment Reduction for Debugging Robotic Systems

Meriel von Stein, Sebastian Elbaum

Complex environments can cause robots to fail. Identifying the key elements of the environment associated with such failures is critical for faster fault isolation and, ultimately, debugging those failures. In this work we present the first automated approach for reducing the environment in which a robot failed. Similar to software… Read more

Download: [Paper] [Repo] [Video]

Probabilistic Conditional System Invariant Generation with Bayesian Inference

Meriel von Stein, Sebastian Elbaum, Lu Feng, Shili Sheng

Probabilistic invariants can encode a family of conditional patterns, are generated using Bayesian inference to leverage observed trace data against priors gleaned from previous experience and expert knowledge, and are ranked based on their surprise value and information content. Our studies on two semi-autonomous mobile robotic systems show how the… Read more

Download: [Paper] [Repo]

Implicit Invariants for Relational Data Structures

Meriel von Stein

Robotics utilizes algorithms for control, localization, and navigation that rely heavily on complex data structures such as matrices and graphs. The values of these data structures are the result of interleaved controllers and complex interactions between equations that are difficult to parameterize as they relate to observed behavior. This makes… Read more

Download: [Presentation] [Paper]