Learning from Computer Simulations to Tackle Real-World Problems

Deep learning has made great strides for problems that can be learned with direct supervision, in which a large dataset with high-quality annotation is provided (e.g., ImageNet). However, collecting such a large dataset is expensive and time-consuming. In this talk, I will discuss our recent works on maximizing the utility of computer simulations to address the data scarcity problem in the real world. I will first present our novel Simulated+Unsupervised (S+U) domain adaptation algorithm, which fully leverages the flexibility of data simulators and bidirectional mappings between synthetic and real data. We show that our approach achieves improved performance on the eye gaze estimation task, outperforming the prior approach by Shrivastava et al. Next, I will introduce our work on building data-driven vehicle collision prediction algorithms. Today’s vehicle collision prediction algorithms are rule-based and have not benefited from the recent developments in deep learning. This is because it is almost impossible to collect a large amount of collision data from the real world. To address this challenge, we collect a large accident dataset using a popular video game named GTA and train end-to-end classification algorithms which identify dangerous objects from the given camera input. Furthermore, we introduce a novel domain adaptation method with which we can further improve the performance of our algorithm in the real world.