dacapo.experiments.trainers

Subpackages

• dacapo.experiments.trainers.gp_augments
  • dacapo.experiments.trainers.gp_augments.augment_config
  • dacapo.experiments.trainers.gp_augments.elastic_config
  • dacapo.experiments.trainers.gp_augments.gamma_config
  • dacapo.experiments.trainers.gp_augments.intensity_config
  • dacapo.experiments.trainers.gp_augments.intensity_scale_shift_config
  • dacapo.experiments.trainers.gp_augments.simple_config
• dacapo.experiments.trainers.optimizers

Submodules

• dacapo.experiments.trainers.dummy_trainer
• dacapo.experiments.trainers.dummy_trainer_config
• dacapo.experiments.trainers.gunpowder_trainer
• dacapo.experiments.trainers.gunpowder_trainer_config
• dacapo.experiments.trainers.trainer
• dacapo.experiments.trainers.trainer_config

Package Contents

Classes

Trainer	Trainer Abstract Base Class
TrainerConfig	A class to represent the Trainer Configurations.
DummyTrainerConfig	This is just a dummy trainer config used for testing. None of the
DummyTrainer	Trainer Abstract Base Class
GunpowderTrainerConfig	This class is used to configure a Gunpowder Trainer. It contains attributes related to trainer type,
GunpowderTrainer	Trainer Abstract Base Class
AugmentConfig	Base class for gunpowder augment configurations. Each subclass of a Augment

class dacapo.experiments.trainers.Trainer

Trainer Abstract Base Class

This serves as the blueprint for any trainer classes in the dacapo library. It defines essential methods that every subclass must implement for effective training of a neural network model.

iteration: int

batch_size: int

learning_rate: float

abstract create_optimizer(model: dacapo.experiments.model.Model) → torch.optim.Optimizer

Creates an optimizer for the model.

Parameters:

model (Model) – The model for which the optimizer will be created.

Returns:

The optimizer created for the model.

Return type:

torch.optim.Optimizer

abstract iterate(num_iterations: int, model: dacapo.experiments.model.Model, optimizer: torch.optim.Optimizer, device: torch.device) → Iterator[dacapo.experiments.training_iteration_stats.TrainingIterationStats]

Performs a number of training iterations.

Parameters:

• num_iterations (int) – Number of training iterations.

• model (Model) – The model to be trained.

• optimizer (torch.optim.Optimizer) – The optimizer for the model.

• device (torch.device) – The device (GPU/CPU) where the model will be trained.

Returns:

An iterator of the training statistics.

Return type:

Iterator[TrainingIterationStats]

abstract can_train(datasets: List[dacapo.experiments.datasplits.datasets.Dataset]) → bool

Checks if the trainer can train with a specific set of datasets.

Some trainers may have specific requirements for their training datasets.

Parameters:

datasets (List[Dataset]) – The training datasets.

Returns:

True if the trainer can train on the given datasets, False otherwise.

Return type:

bool

abstract build_batch_provider(datasets: List[dacapo.experiments.datasplits.datasets.Dataset], model: dacapo.experiments.model.Model, task: dacapo.experiments.tasks.task.Task, snapshot_container: dacapo.store.array_store.LocalContainerIdentifier) → None

Initializes the training pipeline using various components.

This method uses the datasets, model, task, and snapshot_container to set up the training pipeline.

Parameters:

• datasets (List[Dataset]) – The datasets to pull data from.

• model (Model) – The model to inform the pipeline of required input/output sizes.

• task (Task) – The task to transform ground truth into target.

• snapshot_container (LocalContainerIdentifier) – Defines where snapshots will be saved.

class dacapo.experiments.trainers.TrainerConfig

A class to represent the Trainer Configurations.

It is the base class for trainer configurations. Each subclass of a Trainer should have a specific config class derived from TrainerConfig.

name

A unique name for this trainer.

Type:

str

batch_size

The batch size to be used during training.

Type:

int

learning_rate

The learning rate of the optimizer.

Type:

float

name: str

batch_size: int

learning_rate: float

verify() → Tuple[bool, str]

Verify whether this TrainerConfig is valid or not.

Returns:

A tuple containing a boolean indicating whether the TrainerConfig is valid and a message explaining why.

Return type:

tuple

class dacapo.experiments.trainers.DummyTrainerConfig

This is just a dummy trainer config used for testing. None of the attributes have any particular meaning.

trainer_type

mirror_augment: bool

verify() → Tuple[bool, str]

Verify whether this TrainerConfig is valid or not.

Returns:

A tuple containing a boolean indicating whether the TrainerConfig is valid and a message explaining why.

Return type:

tuple

class dacapo.experiments.trainers.DummyTrainer(trainer_config)

Trainer Abstract Base Class

This serves as the blueprint for any trainer classes in the dacapo library. It defines essential methods that every subclass must implement for effective training of a neural network model.

iteration = 0

create_optimizer(model)

Creates an optimizer for the model.

Parameters:

model (Model) – The model for which the optimizer will be created.

Returns:

The optimizer created for the model.

Return type:

torch.optim.Optimizer

iterate(num_iterations: int, model: dacapo.experiments.model.Model, optimizer, device)

Performs a number of training iterations.

Parameters:

• num_iterations (int) – Number of training iterations.

• model (Model) – The model to be trained.

• optimizer (torch.optim.Optimizer) – The optimizer for the model.

• device (torch.device) – The device (GPU/CPU) where the model will be trained.

Returns:

An iterator of the training statistics.

Return type:

Iterator[TrainingIterationStats]

build_batch_provider(datasplit, architecture, task, snapshot_container)

Initializes the training pipeline using various components.

This method uses the datasets, model, task, and snapshot_container to set up the training pipeline.

Parameters:

• datasets (List[Dataset]) – The datasets to pull data from.

• model (Model) – The model to inform the pipeline of required input/output sizes.

• task (Task) – The task to transform ground truth into target.

• snapshot_container (LocalContainerIdentifier) – Defines where snapshots will be saved.

can_train(datasplit)

Checks if the trainer can train with a specific set of datasets.

Some trainers may have specific requirements for their training datasets.

Parameters:

datasets (List[Dataset]) – The training datasets.

Returns:

True if the trainer can train on the given datasets, False otherwise.

Return type:

bool

class dacapo.experiments.trainers.GunpowderTrainerConfig

This class is used to configure a Gunpowder Trainer. It contains attributes related to trainer type, number of data fetchers, augmentations to apply, snapshot interval, minimum masked value, and a boolean value indicating whether to clip raw or not.

trainer_type

This is the type of the trainer which is set to GunpowderTrainer by default.

Type:

class

num_data_fetchers

This is the number of CPU workers who will be dedicated to fetch and process the data.

Type:

int

augments

This is the list of augments to apply during the training.

Type:

List[AugmentConfig]

snapshot_interval

This is the number of iterations after which a new snapshot should be saved.

Type:

Optional[int]

min_masked

This is the minimum masked value.

Type:

Optional[float]

clip_raw

This is a boolean value indicating if the raw data should be clipped to the size of the GT data or not.

Type:

bool

trainer_type

num_data_fetchers: int

augments: List[dacapo.experiments.trainers.gp_augments.AugmentConfig]

snapshot_interval: int | None

min_masked: float | None

clip_raw: bool

class dacapo.experiments.trainers.GunpowderTrainer(trainer_config)

Trainer Abstract Base Class

This serves as the blueprint for any trainer classes in the dacapo library. It defines essential methods that every subclass must implement for effective training of a neural network model.

iteration = 0

create_optimizer(model)

Creates an optimizer for the model.

Parameters:

model (Model) – The model for which the optimizer will be created.

Returns:

The optimizer created for the model.

Return type:

torch.optim.Optimizer

build_batch_provider(datasets, model, task, snapshot_container=None)

Initializes the training pipeline using various components.

This method uses the datasets, model, task, and snapshot_container to set up the training pipeline.

Parameters:

• datasets (List[Dataset]) – The datasets to pull data from.

• model (Model) – The model to inform the pipeline of required input/output sizes.

• task (Task) – The task to transform ground truth into target.

• snapshot_container (LocalContainerIdentifier) – Defines where snapshots will be saved.

iterate(num_iterations, model, optimizer, device)

Performs a number of training iterations.

Parameters:

• num_iterations (int) – Number of training iterations.

• model (Model) – The model to be trained.

• optimizer (torch.optim.Optimizer) – The optimizer for the model.

• device (torch.device) – The device (GPU/CPU) where the model will be trained.

Returns:

An iterator of the training statistics.

Return type:

Iterator[TrainingIterationStats]

next()

can_train(datasets) → bool

Checks if the trainer can train with a specific set of datasets.

Some trainers may have specific requirements for their training datasets.

Parameters:

datasets (List[Dataset]) – The training datasets.

Returns:

True if the trainer can train on the given datasets, False otherwise.

Return type:

bool

class dacapo.experiments.trainers.AugmentConfig

Base class for gunpowder augment configurations. Each subclass of a Augment should have a corresponding config class derived from AugmentConfig.

abstract node(raw_key: gunpowder.ArrayKey, gt_key: gunpowder.ArrayKey, mask_key: gunpowder.ArrayKey) → gunpowder.BatchFilter

return a gunpowder node that performs this augmentation