Skip to main content

Herding Tensor Compilers

Albert Cohen ( Google )

The orchestration of high-performance numerical computations on distributed and heterogeneous systems is not getting any simpler. In the last 5 years, driven by the needs of machine learning, systems and compilers made tremendous progress towards hiding this complexity while delivering excellent performance. These undeniable successes of computing systems and programming language research also came with undesirable and somewhat paradoxical side effects: abstractions and engineering frameworks diversifying out of control while machine learning models got stuck in the rut defined by a small set of highly optimized operators. We will recall algebraic principles supporting the compilation of tensor algebra, and illustrate these principles on three optimization strategies with different degrees of human/expert intervention. While the presentation focuses on optimization and algorithms, we will also discuss MLIR, a large-scale compiler construction effort to rationalize the landscape of machine learning systems.

Speaker bio

Albert is a research scientist at Google. He has been a research scientist at Inria from 2000 to 2018. Alumni of École Normale Supérieure de Lyon and the University of Versailles in 1999. He has been a visiting scholar at the University of Illinois, an invited professor at Philips Research, and a visiting scientist at Facebook Artificial Intelligence Research. Albert Cohen works on parallelizing and optimizing compilers, parallel programming languages and systems, machine learning compilers, synchronous programming, with applications to high-performance computing, artificial intelligence and reactive control. He served as the general or program chair of major conferences, including PLDI, PPoPP, PACT, HiPEAC, CC, the embedded software track of DAC, and as a member of the editorial board of ACM TACO, TECS and IJPP. Several research projects initiated by Albert Cohen resulted in effective transfer to production compilers and programming environments.




Share this: