Storage and Speed

FoldDB optimizes for speed and scalability using:

  • Immutable Atom Storage: Data is stored as immutable atoms, each containing content, metadata, and links to previous versions.
  • AtomRef Pointers: AtomRefs track the latest version of atoms, allowing efficient access without traversing full version histories.
  • Sled Embedded Database: The system uses Sled for persistent, high-performance storage.
  • Append-Only Architecture: New versions create new atoms rather than modifying existing data, preserving historical integrity.

Scaling Considerations

FoldDB scales based on the number of data atoms (N) and schemas (M). Instead of duplicating data across schemas, each data element is stored once as an atom, with schemas referencing atoms through field mappings. This reduces storage requirements, shifting from O(N × M) scaling in traditional systems to O(N) plus a small overhead for schema definitions and mappings.

Processing remains efficient since the system updates references in schemas rather than duplicating data. When a new atom is added, only relevant schema mappings are updated.

The atom storage model provides several performance advantages:

  1. Immutable Data: Atoms remain unchanged after creation, eliminating write concurrency issues.
  2. Version Chains: Atoms link through prev_atom_uuid references, maintaining a full change history.
  3. AtomRef Collections: Organized collections of AtomRefs allow fast retrieval of related data.
  4. Thread-Safe Operations: Atom operations are designed for concurrent execution, ensuring safe multi-threaded access.

This architecture maintains performance and storage efficiency as data volume and schema complexity grow.

← Class Diagram FoldDB Architecture →