Automated Generation of Transformations to Mitigate Sensor Hardware Migration in ADS
Published:
Abstract:Autonomous driving systems (ADSs) rely on massive amounts of sensed data to train their underlying deep neural networks (DNNs). Common sensor hardware migrations can render an existing DNN-dependent pipeline inadequate. This necessitates the development of bespoke transformations to adapt new sensor data to the old trained network, or the retraining of a new network with new sensor data. These solutions are expensive, often performed reactively to sensor hardware migration, and rely only on empirical reconstruction and validation metrics which lack knowledge of the features important to the trained DNN. To address these challenges, we propose PreFixer, a technique that can systematically generate transformations for many types of sensor hardware migration during the ADS development lifecycle. PreFixer collects small datasets using colocated new and old sensors, and then uses that data and the output of the original trained DNN to train an augmented encoder to learn a transformation that maps new sensor data to old sensor data. The trained encoder can then be deployed as a preprocessor to the original trained DNN. Our study shows that, for a common set of camera sensor hardware migrations, PreFixer can match or improve the performance of the best-performing specialized baseline technique in terms of distance travelled safely with 10% of the training dataset, and take at most half of the training time of a new network.
Recommended citation: M. von Stein, H. Wang and S. Elbaum, 'Automated Generation of Transformations to Mitigate Sensor Hardware Migration in ADS,' in IEEE Robotics and Automation Letters, vol. 9, no. 7, pp. 6480-6487, July 2024, doi: 10.1109/LRA.2024.3405810.