dacapo.gp ========= .. py:module:: dacapo.gp Submodules ---------- .. toctree:: :maxdepth: 1 /autoapi/dacapo/gp/copy/index /autoapi/dacapo/gp/dacapo_array_source/index /autoapi/dacapo/gp/dacapo_create_target/index /autoapi/dacapo/gp/dacapo_points_source/index /autoapi/dacapo/gp/elastic_augment_fuse/index /autoapi/dacapo/gp/gamma_noise/index /autoapi/dacapo/gp/product/index /autoapi/dacapo/gp/reject_if_empty/index Classes ------- .. autoapisummary:: dacapo.gp.DaCapoArraySource dacapo.gp.DaCapoTargetFilter dacapo.gp.GammaAugment dacapo.gp.ElasticAugment dacapo.gp.RejectIfEmpty dacapo.gp.CopyMask dacapo.gp.GraphSource dacapo.gp.Product Package Contents ---------------- .. py:class:: DaCapoArraySource(array: dacapo.experiments.datasplits.datasets.arrays.Array, key: gunpowder.ArrayKey) A DaCapo Array source node .. attribute:: array The array to be served. :type: Array .. attribute:: key The key of the array to be served. :type: gp.ArrayKey .. method:: setup() Set up the provider. .. method:: provide(request) Provides the array for the requested ROI. .. note:: This class is a subclass of gunpowder.BatchProvider and is used to serve array data to gunpowder pipelines. .. py:attribute:: array .. py:attribute:: array_spec .. py:attribute:: key .. py:method:: setup() Adds the key and the array spec to the provider. :raises RuntimeError: If the key is already provided. .. rubric:: Examples >>> array_source.setup() .. py:method:: provide(request) Provides data based on the given request. :param request: The request for data :type request: gp.BatchRequest :returns: The batch containing the provided data :rtype: gp.Batch :raises ValueError: If the input data contains NaN values .. rubric:: Examples >>> array_source.provide(request) .. py:class:: DaCapoTargetFilter(predictor: dacapo.experiments.tasks.predictors.Predictor, gt_key: gunpowder.ArrayKey, target_key: Optional[gunpowder.ArrayKey] = None, weights_key: Optional[gunpowder.ArrayKey] = None, mask_key: Optional[gunpowder.ArrayKey] = None) A Gunpowder node for generating the target from the ground truth .. attribute:: Predictor The DaCapo Predictor to use to transform gt into target :type: Predictor .. attribute:: gt The dataset to use for generating the target. :type: ``Array`` .. attribute:: target_key The key with which to provide the target. :type: ``gp.ArrayKey`` .. attribute:: weights_key The key with which to provide the weights. :type: ``gp.ArrayKey`` .. attribute:: mask_key The key with which to provide the mask. :type: ``gp.ArrayKey`` .. method:: setup() Set up the provider. .. method:: prepare(request) Prepare the request. .. method:: process(batch, request) Process the batch. .. note:: This class is a subclass of gunpowder.BatchFilter and is used to generate the target from the ground truth. .. py:attribute:: predictor .. py:attribute:: gt_key .. py:attribute:: target_key .. py:attribute:: weights_key .. py:attribute:: mask_key .. py:attribute:: moving_counts :value: None .. py:method:: setup() Set up the provider. This function sets the provider to provide the target with the given key. :raises RuntimeError: If the key is already provided. .. rubric:: Examples >>> target_filter.setup() .. py:method:: prepare(request) Prepare the request. :param request: The request to prepare. :type request: gp.BatchRequest :returns: The dependencies. :rtype: deps (gp.BatchRequest) :raises NotImplementedError: If the target_key is not provided. .. rubric:: Examples >>> request = gp.BatchRequest() >>> request[gp.ArrayKey("GT")] = gp.ArraySpec(roi=gp.Roi((0, 0, 0), (1, 1, 1))) >>> target_filter.prepare(request) .. py:method:: process(batch, request) Process the batch. :param batch: The batch to process. :type batch: gp.Batch :param request: The request to process. :type request: gp.BatchRequest :returns: The output batch. :rtype: output (gp.Batch) .. rubric:: Examples >>> request = gp.BatchRequest() >>> request[gp.ArrayKey("GT")] = gp.ArraySpec(roi=gp.Roi((0, 0, 0), (1, 1, 1))) >>> target_filter.process(batch, request) .. py:class:: GammaAugment(arrays, gamma_min, gamma_max) Class for applying gamma noise augmentation. .. attribute:: arrays An iterable collection of np arrays to augment .. attribute:: gamma_min A float representing the lower limit of gamma perturbation .. attribute:: gamma_max A float representing the upper limit of gamma perturbation .. method:: setup() Method to configure the internal state of the class .. method:: process() Method to apply gamma noise to the desired arrays .. method:: __augment() Private method to perform the actual augmentation .. py:attribute:: arrays .. py:attribute:: gamma_min .. py:attribute:: gamma_max .. py:method:: setup() Configuring the internal state by iterating over arrays. :raises AssertionError: If the array data type is not float32 or float64 .. rubric:: Examples >>> setup() setup() .. py:method:: process(batch, request) Method to apply gamma noise to the desired arrays. :param batch: The input batch to be processed. :param request: An object which holds the requested output location. :returns: The batch with the gamma noise applied. :raises AssertionError: If the array data type is not float32 or float64 .. rubric:: Examples >>> process(batch, request) process(batch, request) .. py:class:: ElasticAugment(control_point_spacing, control_point_displacement_sigma, rotation_interval, subsample=1, augmentation_probability=1.0, seed=None, uniform_3d_rotation=False) Elasticly deform a batch. Requests larger batches upstream to avoid data loss due to rotation and jitter. :param control_point_spacing: Distance between control points for the elastic deformation, in voxels per dimension. :type control_point_spacing: tuple of int :param control_point_displacement_sigma: Standard deviation of control point displacement distribution, in world coordinates. :type control_point_displacement_sigma: tuple of float :param rotation_interval: Interval to randomly sample rotation angles from (0, 2PI). :type rotation_interval: tuple of two float :param subsample: Instead of creating an elastic transformation on the full resolution, create one sub-sampled by the given factor, and linearly interpolate to obtain the full resolution transformation. This can significantly speed up this node, at the expense of having visible piecewise linear deformations for large factors. Usually, a factor of 4 can safely be used without noticeable changes. However, the default is 1 (i.e., no sub-sampling). :type subsample: int :param seed: Set random state for reproducible results (tests only, do not use in production code!!) :type seed: int :param augmentation_probability: Probability to apply the augmentation. :type augmentation_probability: float :param uniform_3d_rotation: Use a uniform 3D rotation instead of a rotation around a random axis. :type uniform_3d_rotation: bool Provides: * The arrays in the batch, deformed. Requests: * The arrays in the batch, enlarged such that the deformed ROI fits into the enlarged input ROI. Method: setup: Set up the ElasticAugment node. prepare: Prepare the ElasticAugment node. process: Process the ElasticAugment node. .. rubric:: Notes This node is a port of the ElasticAugment node from the original `gunpowder < .. py:attribute:: control_point_spacing .. py:attribute:: control_point_displacement_sigma .. py:attribute:: rotation_start .. py:attribute:: rotation_max_amount .. py:attribute:: subsample .. py:attribute:: augmentation_probability .. py:attribute:: uniform_3d_rotation .. py:attribute:: do_augment :value: False .. py:attribute:: transformations .. py:attribute:: target_rois .. py:method:: setup() Set up the object by calculating the voxel size and spatial dimensions. This method calculates the voxel size by finding the minimum value for each axis from the voxel sizes of all array specs. It then sets the `voxel_size` attribute of the object. The spatial dimensions are also set based on the dimensions of the voxel size. :raises AssertionError: If the voxel size is not a Coordinate object. .. rubric:: Examples >>> setup() .. py:method:: prepare(request) Prepares the request for augmentation. :param request: The request object containing the data to be augmented. :raises AssertionError: If the key in the request is not an ArrayKey or GraphKey. .. rubric:: Examples >>> prepare(request) .. rubric:: Notes This method prepares the request for augmentation by performing the following steps: 1. Logs the preparation details, including the transformation voxel size. 2. Calculates the master ROI based on the total ROI. 3. Generates a uniform random sample and determines whether to perform augmentation based on the augmentation probability. 4. If augmentation is not required, logs the decision and returns. 5. Snaps the master ROI to the grid based on the voxel size and calculates the master transformation. 6. Clears the existing transformations and target ROIs. 7. Iterates over each key in the request and prepares it for augmentation. 8. Updates the upstream request with the modified ROI. .. py:method:: process(batch, request) Process the ElasticAugment node. :param batch: The batch object containing the data to be processed. :param request: The request object specifying the data to be processed. :raises AssertionError: If the key in the request is not an ArrayKey or GraphKey. .. rubric:: Examples >>> process(batch, request) .. rubric:: Notes This method applies the transformation to the data in the batch and restores the original ROIs. .. py:class:: RejectIfEmpty(gt=None, p=0.5, background=0) Reject batches based on the masked-in vs. masked-out ratio. .. attribute:: gt The gt array to use :type: :class:`ArrayKey`, optional .. attribute:: p The probability that we reject until gt is nonempty :type: ``float``, optional Method: setup: Set up the provider. provide: Provide a batch. .. py:attribute:: gt .. py:attribute:: p .. py:attribute:: background :value: 0 .. py:method:: setup() Set up the provider. :raises AssertionError: If only 1 upstream provider is supported. .. rubric:: Examples >>> setup() setup() .. py:method:: provide(request) Provides a batch of data, rejecting empty ground truth (gt) if requested. :param request: The request object containing the necessary information. :returns: The batch of data. :raises AssertionError: If the requested gt is not present in the request. .. rubric:: Examples >>> provide(request) provide(request) .. py:class:: CopyMask(array_key: gunpowder.ArrayKey, copy_key: gunpowder.ArrayKey, drop_channels: bool = False) A class to copy a mask into a new key with the option to drop channels via max collapse. .. attribute:: array_key Original key of the array from where the mask will be copied. :type: gp.ArrayKey .. attribute:: copy_key New key where the copied mask will reside. :type: gp.ArrayKey .. attribute:: drop_channels If True, channels will be dropped via a max collapse. :type: bool .. method:: setup Sets up the filter by enabling autoskip and providing the copied key. .. method:: prepare Prepares the filter by copying the request of copy_key into a dependency. .. method:: process Processes the batch by copying the mask from the array_key to the copy_key. .. note:: This class is a subclass of gunpowder.BatchFilter and is used to copy a mask into a new key with the option to drop channels via max collapse. .. py:attribute:: array_key .. py:attribute:: copy_key .. py:attribute:: drop_channels .. py:method:: setup() Sets up the filter by enabling autoskip and providing the copied key. :raises RuntimeError: If the key is already provided. .. rubric:: Examples >>> copy_mask.setup() .. py:method:: prepare(request) Prepares the filter by copying the request of copy_key into a dependency. :param request: The request to prepare. :returns: The prepared dependencies. :rtype: deps :raises NotImplementedError: If the copy_key is not provided. .. rubric:: Examples >>> request = gp.BatchRequest() >>> request[self.copy_key] = gp.ArraySpec(roi=gp.Roi((0, 0, 0), (1, 1, 1))) >>> copy_mask.prepare(request) .. py:method:: process(batch, request) Processes the batch by copying the mask from the array_key to the copy_key. If "drop_channels" attribute is True, it performs max collapse. :param batch: The batch to process. :param request: The request for processing. :returns: The processed outputs. :rtype: outputs :raises KeyError: If the requested key is not in the request. .. rubric:: Examples >>> request = gp.BatchRequest() >>> request[gp.ArrayKey("ARRAY")] = gp.ArraySpec(roi=gp.Roi((0, 0, 0), (1, 1, 1))) >>> copy_mask.process(batch, request) .. py:class:: GraphSource(key: gunpowder.GraphKey, graph: gunpowder.Graph) A provider for serving graph data in gunpowder pipelines. The Graph Source loads a single graph to serve to the pipeline based on ROI requests it receives. .. attribute:: key The key of the graph to be served. :type: gp.GraphKey .. attribute:: graph The graph to be served. :type: gp.Graph .. method:: setup() Set up the provider. .. method:: provide(request) Provides the graph for the requested ROI. .. note:: This class is a subclass of gunpowder.BatchProvider and is used to serve graph data to gunpowder pipelines. .. py:attribute:: key .. py:attribute:: graph .. py:method:: setup() Set up the provider. This function sets the provider to provide the graph with the given key. :raises RuntimeError: If the key is already provided. .. rubric:: Examples >>> graph_source.setup() .. py:method:: provide(request) Provides the graph for the requested ROI. This method will be passively called by gunpowder to get a batch. Depending on the request we provide a subgraph of our data, or nothing at all. :param request: BatchRequest with the same ROI for :type request: gp.BatchRequest :param each requested array and graph.: :returns: The graph contained in a Batch. :rtype: outputs (gp.Batch) :raises KeyError: If the requested key is not in the request. .. rubric:: Examples >>> request = gp.BatchRequest() >>> request[gp.GraphKey("GRAPH")] = gp.GraphSpec(roi=gp.Roi((0, 0, 0), (1, 1, 1))) >>> graph_source.provide(request) .. py:class:: Product(x1_key: gunpowder.ArrayKey, x2_key: gunpowder.ArrayKey, y_key: gunpowder.ArrayKey) A BatchFilter that multiplies two input arrays and produces an output array. .. attribute:: x1_key The key of the first input array. :type: :class:`ArrayKey` .. attribute:: x2_key The key of the second input array. :type: :class:`ArrayKey` .. attribute:: y_key The key of the output array. :type: :class:`ArrayKey` Provides: y_key (gp.ArrayKey): The key of the output array. Method: __init__: Initialize the Product BatchFilter. setup: Set up the Product BatchFilter. prepare: Prepare the Product BatchFilter. process: Process the Product BatchFilter. .. py:attribute:: x1_key .. py:attribute:: x2_key .. py:attribute:: y_key .. py:method:: setup() Set up the Product BatchFilter. Enables autoskip and specifies the output array. :raises AssertionError: If the input arrays are not provided. .. rubric:: Examples >>> setup() setup() .. py:method:: prepare(request) Prepare the Product BatchFilter. :param request: The batch request. :type request: gp.BatchRequest :returns: The dependencies. :rtype: gp.BatchRequest :raises AssertionError: If the input arrays are not provided. .. rubric:: Examples >>> prepare(request) prepare(request) .. py:method:: process(batch, request) Process the Product BatchFilter. :param batch: The input batch. :type batch: gp.Batch :param request: The batch request. :type request: gp.BatchRequest :returns: The output batch. :rtype: gp.Batch :raises AssertionError: If the input arrays are not provided. .. rubric:: Examples >>> process(batch, request) process(batch, request)