DBToaster: Compiling main-memory database query processors
Yanif Ahmad (Cornell)
Info
|
Date |
6th November 2009 (week 4, Michaelmas Term 2009) |
|
Time |
15:00 |
|
Place |
Lecture Theatre B |
Abstract
We present DBToaster, a novel query compilation framework for producing high performance compiled query executors that incrementally and continuously answer standing aggregate queries using in-memory views. DBToaster's motivating applications include data stream processing, online data warehouses, and financial applications.
DBToaster generates query
engines that perform main-memory processing of standing queries (views), fed by high-volume data streams, by recursively compiling
view maintenance (VM) queries into simple C++ functions for evaluating database updates (deltas).
While today's VM algorithms
consider the impact of single deltas on view queries to produce maintenance queries, we recursively consider deltas of maintenance
queries and compile to thoroughly transform queries into code. Recursive compilation successively elides certain scans and
joins, and eliminates significant query plan interpreter overheads.
We also briefly present ongoing work on the Spread system, a distributed main-memory data warehouse built using DBToaster's recursive compilation algorithm, where rather than producing C++ code, DBToaster produces view maintenance messages that are processed by Spread nodes running in a cluster environment.
Further info
|
Related series |