The problem with `everything is a log` is that is very undisciplined, and trigge...

adityaathalye · 2025-09-04T05:33:32 1756964012

Yes, if the log has no regular schema, and if the overall system has no regular design pattern.

This is the minimal "standard record" that one can use, for all facts pertaining to all domain entities in one's system:

  system_time, valid_time, Entity, Attribute, Value, assert/redact

Now the problem is:

- For small-middle-heavy usage (anything that (reasonably) fits on one machine) how to use existing database technology, so that one can store and query this stuff sufficiently fast. This is what I'm trying to do with SQLite... partly because the "V" of E/A/V benefits from SQLite's "Flexible Typing" system. In SQLite parlance, E, A are TEXT, and V is NUMERIC.

- For at-scale usage (anything that needs many machines), how to make a data system from scratch to store log-structured information efficiently, and how to query it efficiently. See what the Datomic, XTDB, and RedPlanetLabs/Rama people are doing. Essentially: separate storage and compute, event-source everything, and build a system from scratch to use it efficiently and fast at scale.