Abstract
Deep neural networks (DNNs) have developed rapidly and achieved impressive success in various fields recently. Manually constructed DNNs, on the other hand, require prior domain knowledge to build a model. For creating DNNs with less user involvement, evolutionary algorithms (EAs) have attracted increasing attention for generating neural architectures. However, existing evolutionary neural architecture search methods are computationally intensive because the search space is very complex, and the fitness evaluations are computationally prohibitive. Moreover, DNNs are prone to misclassification when encountering adversarial examples that are deliberately designed to be visually imperceptible to humans. Such fragile performance limits the application of DNNs in practical security-related scenarios. Besides, the performance of DNN is also influenced by the hyperparameters during training. A common approach to address the above challenges is adversarial training. Adversarial examples are augmented to the training data to make the trained DNN more robust to adversarial data. Meanwhile, efficiently designing DNNs that are both robust and accurate remains an open challenge.
To address the issues above, this Ph.D. work aims to design neural architectures using multi-objective evolutionary algorithms (MOEAs) to improve the accuracy and robustness simultaneously. This work begins with hyperparameter optimization for convolutional neural networks with a predefined architecture to enhance classification accuracy and robustness. We then propose using an MOEA to search for deep neural architectures robust to five types of well-known adversarial attacks. A normalized error rate of a randomly chosen attack is calculated as the robustness for each newly generated neural architecture at each generation to reduce the computational cost. The evolved DNNs are expected to be significantly more robust against various adversarial attacks and possess a high prediction accuracy. To further speed up the procedure of searching for robust architectures, we finally provide a surrogate-assisted approach to search for robust architectures effectively and efficiently by leveraging low-fidelity and high-fidelity evaluations to predict the performance of architectures.