qrunch.storage.data_persister

Contains manager to handle saving and loading using a serializer and persister.

Classes

DataPersister

Manager to handle saving and loading using a serializer and persister.
class DataPersister

Bases: object

Manager to handle saving and loading using a serializer and persister.

__init__(persister: PersisterProtocol, serializer: SerializerProtocol | None = None) → None

Initialize DataPersister.

Parameters:

  • persister (PersisterProtocol) – An object that can handle reading and writing serialized byte data.

  • serializer (SerializerProtocol | None) – An object that can serialize. Defaults to DataClassJSONSerializer.

Return type:

None

get_all_metadata() → dict[str, dict[str, list[Mapping[str, str | Mapping[str, str | NestedStrDict]]]]]

Return all metadata grouped by key and unique_id.

Discover all persisted (key, unique_id) pairs and load their metadata. For a FilePersister backend, scan the storage directory (or CWD if unset), parse stems that match the internal naming scheme, and for each observed pair delegate decoding to load_metadata(key, unique_id).

Return type:

dict[str, dict[str, list[Mapping[str, str | Mapping[str, str | NestedStrDict]]]]]

has_data(key: str) → bool

Check if the data associated with name exists and contains data.

Parameters:

key (str) – The key for which to find the stored data.

Return type:

bool

has_matching_data(key: str, unique_id: str) → bool

Check if the data associated with name exists and contains data.

Parameters:

  • key (str) – The key for which to find the stored data.

  • unique_id (str) – The provided unique identifier.

Return type:

bool

list_data_for_key(key: str) → list[str]

Return a list of unique_ids for the data stored under the given key for which metadata can be loaded.

Parameters:

key (str) – The key for which to find the stored data.

Returns:

The list of unique ids found for the key.

Return type:

list[str]

load(key: str, unique_id: str, metadata: Mapping[str, str | Mapping[str, str | NestedStrDict]], cls: type[T]) → T | None

Load serialized data using the persister and deserialize it into a dataclass instance.

Parameters:

  • key (str) – The key for which to find the stored data.

  • unique_id (str) – The unique id of describing the data to load.

  • metadata (Mapping[str, str | Mapping[str, str | NestedStrDict]]) – Metadata to save with the instance, for validation if data with matching unique_id is found.

  • cls (type[T]) – The expected dataclass type.

Return type:

T | None

load_metadata(key: str, unique_id: str) → list[Mapping[str, str | Mapping[str, str | NestedStrDict]]]

Load only the metadata associated with a saved entry.

Parameters:

  • key (str) – The key for which to find the stored data.

  • unique_id (str) – The unique id of describing the data to load.

Return type:

list[Mapping[str, str | Mapping[str, str | NestedStrDict]]]

save(key: str, unique_id: str, instance: Any, metadata: Mapping[str, str | Mapping[str, str | NestedStrDict]]) → None

Serialize a dataclass instance to bytes and save it using the persister.

Parameters:

  • key (str) – The key for which to find the stored data.

  • unique_id (str) – The unique id of describing the data to save.

  • instance (Any) – The dataclass instance to serialize and save.

  • metadata (Mapping[str, str | Mapping[str, str | NestedStrDict]]) – Metadata to save with the instance, for validation if data with matching unique_id is found.

Return type:

None