Tensor Comprehensions in SaC: A Minimalistic Notation for High-Performance Computing

Sven-Bodo Scholz ( Radboud University Nijmegen )

4Feb
16:00 4th February 2022 ( week 3, Hilary Term 2022 )
Zoom

This talk focusses on programmer productivity when it comes to defining, understanding, and maintaining computations on multi-dimensional arrays. Shape-invariant programming, i.e., the ability to define APL-like operators that can be applied to arrays of arbitrary dimensionality, surely constitutes a key element here. This raises the question what the minimal building blocks for such operators should be. Should they be a set of fixed primitives a la APL? Should they be a small set of higher-order operators? Should they be inherently n-dimensional or should they be one-dimensional and then be applied recursively? Should they be loops? Or do we need all of these to conveniently express our algorithms?

In the context of SaC, we propose a new form of array comprehensions named "Tensor Comprehensions". This notation strives to be flexible enough to allow for all of the above-mentioned flavours. Despite this flexibility, Tensor Comprehensions aim to be minimalistic in the syntactical requirements, building on sophisticated inference technology to enable programmers to leave out many "obvious" parts. The resulting notation comes rather close to the so-called Tensor Notations used in Physics and Mathematics. As a result, complex operators with rich semantics can be defined more concisely than before.

Speaker bio

Sven-Bodo Scholz is Professor of Computer Science at Radboud University, Nijmegen, Netherlands. He also holds a professorship at Heriot-Watt University, Edinburgh, Scotland. His research is driven by the desire to bridge the gap between high-productivity programming tools and high-performance heterogeneous many-core systems by means of compilation technology. Typical application areas range from multi-sensor robotics systems over big-data analytics to vision and computational science. Target systems range from embedded circuits over large clusters of GPU-accelerated systems into cloud infrastructures. Most of his work on parallelising compiler technology is driven by the needs of industrial project partners such as Intel, AMD, Thales, SAP, Philips and others. Besides regular international dissemination in both, academia and industry, his work has led to several systems in the public domain most notably the SaC compiler tool-chain.