Skip to main content

DISTAL: The Distributed Tensor Algebra Compiler

Rohan Yadav ( Stanford University )

We introduce DISTAL, a compiler for dense tensor algebra that targets modern distributed and heterogenous systems. DISTAL allows users to independently describe how tensors and computation map onto the target machine through the tensors’ formats and a scheduling language. The combination of choices for data and computation distribution creates a design space that includes algorithms from the past (Cannon’s algorithm) and present (COSMA). DISTAL compiles a tensor algebra domain specific language to a distributed task-based runtime system and supports nodes with multi-core CPUs and multiple GPUs. Code generated by DISTAL is competitive with optimized codes for matrix multiply on 256 nodes of the Lassen supercomputer and outperforms existing systems by between 1.8x to 3.7x (with a 45.7x outlier) on higher order tensor operations.

Speaker bio

Rohan Yadav is a second year computer science PhD student at Stanford University, advised by Alex Aiken and Fredrik Kjolstad. He is generally interested in programming languages and computer systems, with a particular focus in systems for parallel and distributed computing.




Share this: