ENTITY Documentation Portal
ENTITY test 17 | storage tradeoffs | 2026-10-01

Storage tradeoff suite

Three fresh, video-recorded tests answer developer objections that a single compression percentage cannot: static compression, exact content deduplication, and archive-vs-live-store behavior.

Bottom line: ENTITY does not beat conventional compressors at static archive size in these tests, and SQLite is faster for indexed lookup in the live-store microbenchmark. ENTITY advantages appear in reusable structured facts, exact duplicate content, compact graph representations, and incremental/provenance-bearing state changes. Treat each result within its stated baseline.

17A — Conventional compression baseline

The same 5,000-record datasets were serialized as canonical NDJSON and compressed losslessly. All compressed streams were decompressed byte-for-byte before PASS. Smaller bytes alone do not imply equivalent query/update semantics.

WorkloadENTITY factoredSQLitegzip-9bzip2-9XZ/LZMA-9
repetitive258,330520,19227,10811,0635,268
mixed385,858401,40850,70529,92636,880
unique1,469,623806,912349,604290,620278,412

Watch the live test video · Result JSON · Runner source

17B — Exact content-addressed deduplication

64 exact 128 KiB objects were ingested at increasing whole-object duplicate ratios. With no duplicates, recipe/metadata overhead made packed state larger than raw bytes. As duplicate content increased, chunk reuse dominated.

Duplicate targetUnique payloadsPacked-state reduction vs rawJournal bytes
064-4.52%9,980,233
254821.47%7,705,843
503247.51%5,402,317
751673.75%3,083,690
90690.12%1,632,567
98198.32%838,084

Watch the live test video · Result JSON · Runner source

17C — Archive vs live-store tradeoff

One 5,000-record mixed dataset was queried 100 times. gzip/XZ lookup intentionally models whole-archive decompression without an external index; timings are illustrative on this workstation only.

PathMedian lookup (µs)p95 (µs)
SQLite256.2318.6
ENTITY entity_view2954.13642.5
gzip full decompress2216.42984.9
XZ full decompress4524.95279.2

Single-record update: ENTITY appended 1,112 journal bytes and passed restart verification. Rebuilding the standalone compressed archive produced 50,765 gzip bytes or 37,104 XZ bytes.

Watch the live test video · Result JSON · Runner source

Evidence

Machine-readable Test 17 bundle · Canonical Git-byte SHA-256 manifest

Claim boundary: these are measured/specified engineering scenarios, not universal compression, database replacement or production performance claims.