Skip to main content

Group-equivariance for data-efficient deep learning in Earth Observation

Supervisors

Suitable for

MSc in Advanced Computer Science

Abstract

Summary: Leverage known symmetries in satellite data, e.g. rotation, flips, scaling, for more data-efficient learning of downstream tasks. Abstract: Choosing the right inductive bias in machine learning tasks can reduce the amount of data required for training by orders of magnitude. One inductive bias that is ubiquitous in computer vision tasks is shift-invariance (e.g. classification) and shift-equivariance (e.g. single/multi image super-resolution, segmentation, image translation, detection), that is, there is a bijection between shifts in the input domain and shifts in the output co-domain. The success of deep learning in computer vision is owed to these inductive biases being baked into an architecture through CNN layers, which in theory allows them to detect the same feature anywhere in an image, under any shift. Rotations and flips and scalings are also desirable symmetries for low-level features (eg. oriented edges and textures), but ones that must be enforced on CNNs, usually through data augmentation techniques, at the expense of model size and training time. Group-equivariant CNNs (g-CNN) [Cohen & Welling, 2016] were the first generalization of CNNs, with the property of exact equivariance in the group of 90-degree rotations, flips and translations (the p4m group). This type of architecture can be even more effective for downstream tasks common in Earth Observation (land cover classification [Marcos et al. 2018], building segmentation, multi-frame super-resolution), because objects in satellite imagery, like coastlines and rivers, can appear under any orientation. This project will explore and leverage known symmetries in satellite data, e.g. rotation, flips, scaling, permutation-invariance (in multi-image setups) for more data-efficient learning of common downstream tasks. References: Cohen, T. and Welling, M., 2016, June. Group equivariant convolutional networks. In International conference on machine learning (pp. 2990-2999). Marcos, D., Volpi, M., Kellenberger, B. and Tuia, D., 2018. Land cover mapping at very high resolution with rotation equivariant CNNs: Towards small yet accurate models. ISPRS journal of photogrammetry and remote sensing, 145, pp.96-107.

Prerequisites
Strong Python coding, experience with deep learning and PyTorch for computer vision, good understanding of CNNs. Experience with Git.
Prerequisites
Strong Python coding, experience with deep learning and PyTorch for computer vision, good understanding of CNNs. Experience with Git.
Desirable: Experience with satellite imagery, exposure in basic group theory.