Reinforcement Learning as an Alternative to Reachability Analysis for Falsification of AD Functions

T. Johansson, A.M. Acosta, A. Schliep and P. Falcone

In Dec 2021. To appear at ML4AD (NeurIPS workshop on Autonomous Driving).

chalmers.seAbstractReachability analysis (RA) is one of the classical approaches to study the safety ofautonomous systems, for example through falsification, the identification of initialsystem states which can under the right disturbances lead to unsafe or undesirableoutcome states. The advantage of obtaining exact answers via RA requires analyti-cal system models often unavailable for simulation environments for autonomousdriving (AD) systems. RA suffers from rapidly rising computational costs as thedimensionality increases and ineffectiveness in dealing with nonlinearities such assaturation. Here we present an alternative in the form of a reinforcement learning(RL) approach which empirically shows good agreement with RA falsificationfor an Adaptive Cruise Controller, it can deal with saturation, and, in preliminarydata, compares favorably in computational effort against RA. Due to the choiceof reward function, the RL’s estimated value function provides insights into theease of causing unsafe outcomes and allows for direct comparison with the RAfalsification results.