dacapo.store.local_weights_store

Attributes

logger

Classes

LocalWeightsStore

A local store for network weights.

Module Contents

dacapo.store.local_weights_store.logger

class dacapo.store.local_weights_store.LocalWeightsStore(basedir)

A local store for network weights.

All weights are stored in a directory structure like this:

` basedir ├── run1 │   ├── checkpoints │   │   ├── iterations │   │   │   ├── 0 │   │   │   ├── 1 │   │   │   ├── ... │   ├── dataset1 │   │   ├── criterion1.json │   ├── dataset2 │   │   ├── criterion2.json ├── run2 │   ├── ... `

basedir

The base directory where the weights are stored.

latest_iteration()

Return the latest iteration for which weights are available for the given run.

store_weights()

Store the network weights of the given run.

retrieve_weights()

Retrieve the network weights of the given run.

remove()

Remove the network weights of the given run.

store_best()

Store the best weights in a easy to find location.

retrieve_best()

Retrieve the best weights of the given run.

Note

The weights are stored in the format of a Weights object, which is a simple container for the model and optimizer state dicts.

basedir

latest_iteration(run: str) → int | None

Return the latest iteration for which weights are available for the given run.

Parameters:

run – The name of the run.

Returns:

The latest iteration for which weights are available, or None if no weights are available.

Raises:

FileNotFoundError – If the run directory does not exist.

Examples

>>> store.latest_iteration("run1")

Note

The iteration is determined by the number of the subdirectories in the “iterations” directory.

store_weights(run: dacapo.experiments.run.Run, iteration: int)

Store the network weights of the given run.

Parameters:

• run – The run object.

• iteration – The iteration number.

Raises:

FileNotFoundError – If the run directory does not exist.

Examples

>>> store.store_weights(run, 0)

Note

The weights are stored in the format of a Weights object, which is a simple container for the model and optimizer state dicts.

retrieve_weights(run: str, iteration: int) → dacapo.store.weights_store.Weights

Retrieve the network weights of the given run.

Parameters:

• run – The name of the run.

• iteration – The iteration number.

Returns:

The network weights.

Raises:

FileNotFoundError – If the weights file does not exist.

Examples

>>> store.retrieve_weights("run1", 0)

Note

The weights are stored in the format of a Weights object, which is a simple container for the model and optimizer state dicts.

remove(run: str, iteration: int)

Remove the weights for a specific run and iteration.

Parameters:

• run (str) – The name of the run.

• iteration (int) – The iteration number.

Raises:

FileNotFoundError – If the weights file does not exist.

Examples

>>> store.remove("run1", 0)

Note

The weights are stored in the format of a Weights object, which is a simple container for the model and optimizer state dicts.

store_best(run: str, iteration: int, dataset: str, criterion: str)

Store the best weights in a easy to find location. Symlinks weights from appropriate iteration # TODO: simply store a yaml of dataset/criterion -> iteration/parameter id

Parameters:

• run (str) – The name of the run.

• iteration (int) – The iteration number.

• dataset (str) – The name of the dataset.

• criterion (str) – The criterion for selecting the best weights.

Raises:

FileNotFoundError – If the weights file does not exist.

Examples

>>> store.store_best("run1", 0, "dataset1", "criterion1")

Note

The best weights are stored in a json file that contains the iteration number.

retrieve_best(run: str, dataset: str | dacapo.experiments.datasplits.datasets.dataset.Dataset, criterion: str) → int

Retrieve the best weights for a given run, dataset, and criterion.

Parameters:

• run (str) – The name of the run.

• dataset (str | Dataset) – The name of the dataset or a Dataset object.

• criterion (str) – The criterion for selecting the best weights.

Returns:

The iteration number of the best weights.

Return type:

int

Raises:

FileNotFoundError – If the weights file does not exist.

Examples

>>> store.retrieve_best("run1", "dataset1", "criterion1")

Note

The best weights are stored in a json file that contains the iteration number.