Abstract
The next generation of autonomous vehicles will provide major improvements in traffic flow, fuel efficiency, and vehicle safety. Several challenges currently prevent the deployment of autonomous vehicles, one aspect of which is robust and adaptable vehicle control. Designing a controller for autonomous vehicles capable of providing adequate performance in all driving scenarios is challenging due to the highly complex environment and inability to test the system in the wide variety of scenarios which it may encounter after deployment.
Deep learning methods have shown great promise in not only providing excellent performance for complex and non-linear control problems, but also in generalising previously learned rules to new scenarios. For these reasons, the use of deep neural networks for vehicle control has gained significant interest. However, the opaqueness of deep neural networks means that traditional validation and verification techniques cannot provide the safety guarantees required to deploy these models in safety-critical systems such as autonomous vehicles. Therefore, new techniques for ensuring the safety of deep neural network-driven autonomous vehicles are required. For these reasons, techniques utilising deep neural networks for autonomous vehicle control that ensure the safety of the vehicle are investigated in this thesis.
First, techniques based on imitation learning and reinforcement learning are utilised to learn control policies for autonomous highway driving. The robustness and generalisability of different models are compared in simulated highway driving scenarios to investigate the effectiveness of learning algorithms. To address the safety and interpretability of the vehicle motion control system, the vehicle safety is enhanced through rule-based and interpretable safety cages which intervene on the actions of the learned control policies. Furthermore, these interventions by the safety cages are also utilised in re-training less safe control policies, improving the safety and robustness of the final control policy.
Second, the validation of deep neural network based motion control algorithms is addressed. Since autonomous vehicles may encounter a wide variety of possible scenarios on the road and testing the control policy in all possible scenarios is infeasible, more efficient validation strategies are required. As most collisions happen in rare edge-cases that are seen rarely during normal driving, this work proposed the use of learned adversarial agents to generate these safety-critical edge cases. Adversarial agents are trained through reinforcement learning to act as agents in the same environment as the target control policy, and act in such a way that the target vehicle causes a collision. It is demonstrated that in this way, collision cases are generated more efficiently, and by imposing limits on the actions of the adversaries, all collisions would have been preventable and therefore represent a safety-critical weakness in the target control policy under testing. The proposed approach is shown to reveal weaknesses in the tested policies that were not apparent in previous testing frameworks, with over 11,000 collision cases found.
Third, the use of adversarial learning to improve the robustness of the target policies is investigated. Once the adversarial agents have demonstrated that weaknesses exist in these control policies, the next natural step is to correct these vulnerabilities. Two techniques leveraging the information from the adversarial testing are proposed. The first technique is based on minimax adversarial reinforcement learning, whilst the second technique leverages mixture density networks and imitation learning. It is shown that both of these techniques can be utilised to improve the safety and robustness of the vehicle control policies.