Thursday, February 14, 2019

RocksDB: cloud native, embedded db engine (with LSM trees)

Rocksdb-icon.svgRocksDB - Wikipedia

"RocksDB is a high performance[1][2][3][4][5] embedded database for key-valuedata. It is a fork of LevelDB by Facebook optimized to exploit many central processing unit (CPU) cores, and make efficient use of fast storage, such as solid-state drives (SSD), for input/output (I/O) bound workloads. It is based on a log-structured merge-tree (LSM tree) data structure. It is written in C++ and provides official application programming interface (API) language bindings for C++, C, and Java; alongside many third-party language bindings. RocksDB is open-source software..."

Podcast interview with great technical description of this DB engine:

RocksDB with Dhruba Borthakur and Igor Canadi - Software Engineering Daily

RocksDB was created in Facebook, as a fork of Google's LevelDB that is used as embedded DB of Chrome web browser.

RocksDB is different from typical databases that are using standard b-tree for storing indexes.
Instead, it is using "LSM" data storage, that is optimized for SSD and "cloud" storage.

compaction of data in a Log-Structured Merge tree

LSM trees are used in data stores such as Bigtable, HBase, LevelDB, MongoDB, SQLite4[5], Tarantool [6], RocksDB, WiredTiger,[7] Apache Cassandra, and InfluxDB.[8]

RocksDB is used in combination with various database front-ends:

  • MyRocks = MySQL + RocksDB engine
  • MongoRocks = MongoDB + RocksDB engine
  • Cassandra + RocksDB
  • ...

Company Rockset is providing products and services related to RocksDB.