Comparison of Deep Learning Architectures for Nonlinear System Identification[go to master theses]
Open Master Thesis - Contact a supervisor for more details!
System identification deals with deriving mathematical models of dynamical systems from data. Traditionally, this is a costly and challenging task, as it requires the collection of data via experiments and model tests, as well as domain knowledge for tuning of the identified model. With the wide-spread use of sensors in various systems, such as vehicles and industrial plants, the application of deep neural networks to the identification task becomes feasible. Deep neural networks are capable of learning complex nonlinear dynamics from large quantities of data, which has been shown in related domains such as predictive maintenance and time-series forecasting.
The current state-of-the-art in system identification of vehicles, such as unmanned aerial or surface vehicles, are recurrent and convolutional neural networks  . Recently, purely attention-based architectures, such as transformers  , have established a new state-of-the-art in many natural language processing tasks thereby replacing the former network types. Following up on this development, it is of interest whether transformers also prove useful for modeling nonlinear dynamical systems.
The task of the Master thesis is the evaluation of various deep learning architectures for the identification of helicopter dynamics (dataset). The student will adapt the transformer architecture for modeling dynamical systems, and compare its performance to recurrent and convolutional architectures. This work fills a gap in the current related work, as most papers only consider a single class of deep neural network during evaluation.
- Devlin, Jacob, et al. “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding.” arXiv preprint arXiv:1810.04805 (2018). https://arxiv.org/abs/1810.04805
- Lopez, Mario, and Wen Yu. “Nonlinear system modeling using convolutional neural networks.” 2017 14th International Conference on Electrical Engineering, Computing Science and Automatic Control (CCE). IEEE, 2017. https://ieeexplore.ieee.org/document/8108894
- Mohajerin, Nima, Melissa Mozifian, and Steven Waslander. “Deep Learning a Quadrotor Dynamic Model for Multi-Step Prediction.” 2018 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2018.https://ieeexplore.ieee.org/document/8460840
- Vaswani, Ashish, et al. “Attention Is All You Need.” Advances in Neural Information Processing Systems. 2017.https://arxiv.org/abs/1706.03762