I-Store: Data Management on Fast Networks
The next generation of high-performance RDMA-capable networks requires a fundamental rethinking of the design of modern distributed in-memory DBMSs. These systems are commonly designed under the assumption that the network is the bottleneck and thus must be avoided as much as possible. This assumption no longer holds true. With InfiniBand FDR 4x, the bandwidth available to transfer data across the network is in the same ballpark as the bandwidth of one memory channel, and the bandwidth increases even more with the most recent EDR standard. Moreover, with increasing advances in RDMA, transfer latencies improve similarly fast.
In this project, we first argue that the “old” distributed database design is not capable of taking full advantage of fast networks and suggest a new architecture. We currently implement a new database system called I-Store which is based on a new architecture called Network-attached Memory Architecture, which separates compute storage from storage. The NAM architecture has not only the advantage that compute storage can leverage the main memory of other machines but we can also better size the system with regard to the resource needs (compute vs. storage) and can provide better load-balancing.
- Abdallah Salama, Carsten Binnig, Tim Kraska, Ansgar Scherp, Tobias Ziegler:
Rethinking Distributed Query Execution on High-Speed Networks. IEEE Data Eng. Bull. 40(1): 27-37 (2017)
- Carsten Binnig, Erfan Zamanian, Tim Harris, Tim Kraska: The End of a Myth: Distributed Transaction can Scale. Research Paper, PVLDB 2017
- Carsten Binnig, Andrew Crotty, Alex Galakatos et al.: The End of Slow Networks: It’s Time for a Redesign. Vision Paper, PVLDB 2016
- Carsten Binnig, Erfan Zamanian, Tim Kraska et al.:Making Distributed Transactions Scale. search Paper, Nedbday 2016
- Carsten Binnig, Ugur Cetintemel, Tim Kraska et al.: I-Store: Data Management for Fast Networks. Research Paper, Nedbday 2015