XDB – A novel Database Architecture for Data Analytics as a Service
Parallel shared-nothing database systems are major platforms for efficiently analyzing large amounts of structured data. However, in order to offer SQL-like services for data analytics in the cloud, providers such as Amazon and Google do not use these systems as a basis. A major reason for this trend is that existing parallel shared-nothing database systems are expensive and that they do not fulfill many of the requirements such as elasticity and fault-tolerance needed for providing a service for data analytics in the cloud.
In this paper, we present an overview of an elastic and fault-tolerant database system called XDB, which supports complex analytics. XDB builds on the following novel concepts:
- a locality-aware partitioning scheme that supports elasticity with regard to data and queries,
- a cost-based fault-tolerance scheme that allows to recover from mid-query faults, and
- adaptive parallelization techniques to better support complex analytical queries.
XDB is implemented using a middleware approach on top of multiple nodes each hosting an instance of a single node database system (MySQL in our prototype).
Initial experiments show that our novel concepts effectively support elasticity, fault-tolerance and complex analytics when compared to the traditional behavior of existing databases.
- Carsten Binnig, Abdallah Salama, Erfan Zamanian, Muhammad El-Hindi, Sebastian Feil, Tobias Ziegler: Spotgres – parallel data analytics on Spot Instances. ICDE Workshops 2015: 14-21
- Erfan Zamanian, Carsten Binnig, Abdallah Salama: Locality-aware Partitioning in Parallel Database Systems. SIGMOD Conference 2015: 17-30
- Abdallah Salama, Carsten Binnig, Tim Kraska, Erfan Zamanian: Cost-based Fault-tolerance for Parallel Data Processing. SIGMOD Conference 2015: 285-297
- Carsten Binnig, Abdallah Salama, Erfan Zamanian, Harald Kornmayer, Sven Listing, Alexander C. Müller: XDB – A Novel Database Architecture for Data Analytics as a Service. IEEE BigData Congress 2014: 96-103 (Best Paper)
- Carsten Binnig, Abdallah Salama, Erfan Zamanian: DoomDB: kill the query. SIGMOD Conference 2014: 913-916 (Best Demo)
- Carsten Binnig, Abdallah Salama, Alexander C. Müller, Erfan Zamanian, Harald Kornmayer, Sven Listing: XDB: a novel database architecture for data analytics as a service. SoCC 2013: 39:1-39:2