Storage

The Storage object owns the on-disk artefact that holds a reconstructed network. It is reachable from any Scaffold via scaffold.storage, and is parameterised by two things:

  • an engine (chosen via the storage.engine configuration key) that picks the on-disk format;

  • a root (chosen via storage.root), the path the engine writes to.

The engine is loaded from the bsb.storage.engines plugin category (see Plugins); two engines ship by default, fs and hdf5.

Storage engines

FS

A lightweight filesystem-backed engine (FileSystemEngine). The root is a directory containing a metadata.json plus two subfolders:

  • files/: opaque files stored by the FileStore (typically the active configuration JSON and any user-attached blobs).

  • file_meta/: sidecar metadata for each file (mtime, encoding, content hash, producer).

The FS engine does not store placement or connectivity data; those raise NotImplementedError on this backend. It exists for workflows that only need the configuration and file-store side of a storage object, for example distributing a config bundle without a compiled network.

HDF5

The full storage backend (HDF5Engine), used for compiled networks. The root is a single HDF5 file. Top-level layout:

  • /placement/<cell_type>: one group per PlacementSet, containing the per-chunk position, rotation, morphology and label datasets.

  • /connectivity/<tag>: one group per ConnectivitySet, holding incoming and outgoing per-chunk connection blocks.

  • /files/<uuid>: file store (active configuration JSON and any user blobs), with per-file meta stored as a JSON-encoded meta attribute on each dataset.

  • /morphologies: the MorphologyRepository, content-addressed by hash in the morphology metadata.

Concurrent reader safety, locking, and slow-lock diagnostics are handled by the engine; component code does not need to manage these.

The on-disk artefact also carries a provenance bundle (storage identity, plugin manifest, host info, timestamps, per-PlacementSet revision counters, …). Plugin authors and operators that need the full bundle layout should consult Storage engines.