Skip to main content

Automatically synthesizing Spark programs

Supervisor

Amir Shaikhha
(Research Associate, University College Research Associate, University College)

Suitable for

MSc in Advanced Computer Science
Computer Science, Part B
Mathematics and Computer Science, Part C
Computer Science and Philosophy, Part C
Computer Science, Part C

Abstract

There are different ways to write a query processing algorithm (such as a join algorithm) using Spark. For different data sizes and based on the number of worker nodes, the best Spark implementation may vary. The aim of this project is to use program synthesis techniques to automatically derive the most efficient Spark program based on the given data information and the given platform.