Sunday, October 29, 2023

sqlite-zstd: row-level compression for SQLite

 sqlite-zstd: Transparent dictionary-based row-level compression for SQLite - An sqlite extension written in Rust to reduce the database size without losing functionality - phiresky's blog

data is a json blob from the perspective of SQLite, but it is parsed into a strictly typed structure in the Rust application code.

phiresky/sqlite-zstd: Transparent dictionary-based row-level compression for SQLite

@GitHub, Rust, LGPL

Depending on the data, this can reduce the size of the database by 80% while keeping performance mostly the same (or even improving it, since the data to be read from disk is smaller).

Note that a compression VFS such as https://github.com/mlin/sqlite_zstd_vfs might be suited better depending on the use case. That has very different tradeoffs and capabilities, but the end result is similar.

mlin/sqlite_zstd_vfs: SQLite3 extension for read/write storage compression with Zstandard



Never Too Rich Or Thin: Compress Sqlite 80% | Hackaday



No comments: