dacapo.utils.pipeline

Module Contents

Classes

CreatePoints	Convenience wrapper for BatchProviders with
MakeRaw	Convenience wrapper for BatchProviders with
DilatePoints	Convenience wrapper for BatchProviders with
RandomDilateLabels	Convenience wrapper for BatchProviders with
Relabel	Convenience wrapper for BatchProviders with
ExpandLabels	Convenience wrapper for BatchProviders with
ZerosSource	Superclass for all nodes in a gunpowder graph.

Functions

random_source_pipeline([voxel_size, input_shape, ...])

Create a random source pipeline and batch request for example training.

class dacapo.utils.pipeline.CreatePoints(labels, num_points=(20, 150))

Convenience wrapper for BatchProviders with exactly one input provider.

By default, a node of this class will expose the same ProviderSpec as the upstream provider. You can modify the provider spec by calling provides() and updates() in setup().

Subclasses need to implement at least process() to modify a passed batch (downstream). Optionally, the following methods can be implemented:

setup()

Initialize this filter. Called after setup of the DAG. All upstream providers will be set up already.

teardown()

Destruct this filter, free resources, stop worker processes.

prepare()

Prepare for a batch request. Always called before each process(). Used to communicate dependencies.

process(batch, request)

To be implemented in subclasses.

Filter a batch, will be called after prepare(). Should return a Batch containing modified Arrays and Graphs. Keys in the returned batch will replace the associated data in the original batch. If None is returned it is assumed that the batch has been modified in place. request is the same as passed to prepare(), provided for convenience.

Parameters:

• batch (Batch) – The batch received from upstream to be modified by this node.

• request (BatchRequest) – The request this node received. The updated batch should meet this request.

class dacapo.utils.pipeline.MakeRaw(raw, labels, gaussian_noise_args: Iterable = (0.5, 0.1), gaussian_noise_lim: float = 0.3, gaussian_blur_args: Iterable = (0.5, 1.5), membrane_like=True, membrane_size=3, inside_value=0.5)

Convenience wrapper for BatchProviders with exactly one input provider.

By default, a node of this class will expose the same ProviderSpec as the upstream provider. You can modify the provider spec by calling provides() and updates() in setup().

Subclasses need to implement at least process() to modify a passed batch (downstream). Optionally, the following methods can be implemented:

setup()

Initialize this filter. Called after setup of the DAG. All upstream providers will be set up already.

teardown()

Destruct this filter, free resources, stop worker processes.

prepare()

Prepare for a batch request. Always called before each process(). Used to communicate dependencies.

setup()

To be implemented in subclasses.

Called during initialization of the DAG. Callees can assume that all upstream providers are set up already.

In setup, call provides() or updates() to announce the arrays and points provided or changed by this node.

process(batch, request)

To be implemented in subclasses.

Filter a batch, will be called after prepare(). Should return a Batch containing modified Arrays and Graphs. Keys in the returned batch will replace the associated data in the original batch. If None is returned it is assumed that the batch has been modified in place. request is the same as passed to prepare(), provided for convenience.

Parameters:

• batch (Batch) – The batch received from upstream to be modified by this node.

• request (BatchRequest) – The request this node received. The updated batch should meet this request.

class dacapo.utils.pipeline.DilatePoints(labels, dilations=[2, 8])

Convenience wrapper for BatchProviders with exactly one input provider.

By default, a node of this class will expose the same ProviderSpec as the upstream provider. You can modify the provider spec by calling provides() and updates() in setup().

Subclasses need to implement at least process() to modify a passed batch (downstream). Optionally, the following methods can be implemented:

setup()

Initialize this filter. Called after setup of the DAG. All upstream providers will be set up already.

teardown()

Destruct this filter, free resources, stop worker processes.

prepare()

Prepare for a batch request. Always called before each process(). Used to communicate dependencies.

process(batch, request)

To be implemented in subclasses.

Filter a batch, will be called after prepare(). Should return a Batch containing modified Arrays and Graphs. Keys in the returned batch will replace the associated data in the original batch. If None is returned it is assumed that the batch has been modified in place. request is the same as passed to prepare(), provided for convenience.

Parameters:

• batch (Batch) – The batch received from upstream to be modified by this node.

• request (BatchRequest) – The request this node received. The updated batch should meet this request.

class dacapo.utils.pipeline.RandomDilateLabels(labels, dilations=[2, 8])

Convenience wrapper for BatchProviders with exactly one input provider.

By default, a node of this class will expose the same ProviderSpec as the upstream provider. You can modify the provider spec by calling provides() and updates() in setup().

Subclasses need to implement at least process() to modify a passed batch (downstream). Optionally, the following methods can be implemented:

setup()

Initialize this filter. Called after setup of the DAG. All upstream providers will be set up already.

teardown()

Destruct this filter, free resources, stop worker processes.

prepare()

Prepare for a batch request. Always called before each process(). Used to communicate dependencies.

process(batch, request)

To be implemented in subclasses.

Filter a batch, will be called after prepare(). Should return a Batch containing modified Arrays and Graphs. Keys in the returned batch will replace the associated data in the original batch. If None is returned it is assumed that the batch has been modified in place. request is the same as passed to prepare(), provided for convenience.

Parameters:

• batch (Batch) – The batch received from upstream to be modified by this node.

• request (BatchRequest) – The request this node received. The updated batch should meet this request.

class dacapo.utils.pipeline.Relabel(labels, connectivity=1)

Convenience wrapper for BatchProviders with exactly one input provider.

By default, a node of this class will expose the same ProviderSpec as the upstream provider. You can modify the provider spec by calling provides() and updates() in setup().

Subclasses need to implement at least process() to modify a passed batch (downstream). Optionally, the following methods can be implemented:

setup()

Initialize this filter. Called after setup of the DAG. All upstream providers will be set up already.

teardown()

Destruct this filter, free resources, stop worker processes.

prepare()

Prepare for a batch request. Always called before each process(). Used to communicate dependencies.

process(batch, request)

To be implemented in subclasses.

Filter a batch, will be called after prepare(). Should return a Batch containing modified Arrays and Graphs. Keys in the returned batch will replace the associated data in the original batch. If None is returned it is assumed that the batch has been modified in place. request is the same as passed to prepare(), provided for convenience.

Parameters:

• batch (Batch) – The batch received from upstream to be modified by this node.

• request (BatchRequest) – The request this node received. The updated batch should meet this request.

class dacapo.utils.pipeline.ExpandLabels(labels, background=0)

Convenience wrapper for BatchProviders with exactly one input provider.

By default, a node of this class will expose the same ProviderSpec as the upstream provider. You can modify the provider spec by calling provides() and updates() in setup().

Subclasses need to implement at least process() to modify a passed batch (downstream). Optionally, the following methods can be implemented:

setup()

Initialize this filter. Called after setup of the DAG. All upstream providers will be set up already.

teardown()

Destruct this filter, free resources, stop worker processes.

prepare()

Prepare for a batch request. Always called before each process(). Used to communicate dependencies.

process(batch, request)

To be implemented in subclasses.

Filter a batch, will be called after prepare(). Should return a Batch containing modified Arrays and Graphs. Keys in the returned batch will replace the associated data in the original batch. If None is returned it is assumed that the batch has been modified in place. request is the same as passed to prepare(), provided for convenience.

Parameters:

• batch (Batch) – The batch received from upstream to be modified by this node.

• request (BatchRequest) – The request this node received. The updated batch should meet this request.

class dacapo.utils.pipeline.ZerosSource(key, spec)

Superclass for all nodes in a gunpowder graph.

A BatchProvider provides Batches containing Arrays and/or Graph. The available data is specified in a ProviderSpec instance, accessible via spec.

To create a new node, subclass this class and implement (at least) setup() and provide().

A BatchProvider can be linked to any number of other BatchProviders upstream. If your node accepts exactly one upstream provider, consider subclassing BatchFilter instead.

setup()

To be implemented in subclasses.

Called during initialization of the DAG. Callees can assume that all upstream providers are set up already.

In setup, call provides() to announce the arrays and points provided by this node.

provide(request)

To be implemented in subclasses.

This function takes a BatchRequest and should return the corresponding Batch.

Parameters:

request (BatchRequest) – The request to process.

dacapo.utils.pipeline.random_source_pipeline(voxel_size=(8, 8, 8), input_shape=(148, 148, 148), dtype=np.uint8, expand_labels=False, relabel_connectivity=1, random_dilate=True, num_points=(20, 150), gaussian_noise_args=(0, 0.1), gaussian_blur_args=(0.5, 1.5), membrane_like=True, membrane_size=3, inside_value=0.5)

Create a random source pipeline and batch request for example training.

Parameters:

• voxel_size (tuple of int) – The size of a voxel in world units.

• input_shape (tuple of int) – The shape of the input arrays.

• dtype (numpy.dtype) – The dtype of the label arrays.

• expand_labels (bool) – Whether to expand the labels into the background.

• relabel_connectivity (int) – The connectivity used for for relabeling.

• random_dilate (bool) – Whether to randomly dilate the individual labels.

• num_points (tuple of int) – The range of the number of points to add to the labels.

• gaussian_noise_args (tuple of float) – The mean and standard deviation of the Gaussian noise to add to the raw array.

• gaussian_blur_args (tuple of float) – The mean and standard deviation of the Gaussian blur to apply to the raw array.

• membrane_like (bool) – Whether to generate a membrane-like structure in the raw array.

• membrane_size (int) – The width of the membrane-like structure on the outside of the objects.

• inside_value (float) – The value to set inside the membranes of objects.

Returns:

The batch generating Gunpowder pipeline. gunpowder.BatchRequest: The batch request for the pipeline.

Return type:

gunpowder.Pipeline