dacapo.blockwise

Submodules

Classes

DaCapoBlockwiseTask

A task to run a blockwise worker function. This task is used to run a

Functions

run_blockwise(worker_file, total_roi, read_roi, write_roi)

Run a function in parallel over a large volume.

segment_blockwise(segment_function_file, context, ...)

Run a segmentation function in parallel over a large volume.

Package Contents

class dacapo.blockwise.DaCapoBlockwiseTask(worker_file: str | upath.UPath, total_roi: daisy.Roi, read_roi: daisy.Roi, write_roi: daisy.Roi, num_workers: int = 16, max_retries: int = 2, timeout=None, upstream_tasks=None, *args, **kwargs)

A task to run a blockwise worker function. This task is used to run a blockwise worker function on a given ROI.

worker_file

The path to the worker file.

Type:

str | Path

total_roi

The ROI to process.

Type:

Roi

read_roi

The ROI to read from for a block.

Type:

Roi

write_roi

The ROI to write to for a block.

Type:

Roi

num_workers

The number of workers to use.

Type:

int

max_retries

The maximum number of times a task will be retried if failed (either due to failed post check or application crashes or network failure)

Type:

int

timeout

The timeout for the task.

upstream_tasks

The upstream tasks.

\*args

Additional positional arguments to pass to worker_function.

\*\*kwargs

Additional keyword arguments to pass to worker_function.

__init__()

Initialize the task.

worker_name
worker
timestamp
task_id
process_function
read_write_conflict
fit
kwargs
dacapo.blockwise.run_blockwise(worker_file: str | upath.UPath, total_roi: funlib.geometry.Roi, read_roi: funlib.geometry.Roi, write_roi: funlib.geometry.Roi, num_workers: int = 16, max_retries: int = 1, timeout=None, upstream_tasks=None, *args, **kwargs)

Run a function in parallel over a large volume.

Parameters:

  • worker_file (str or Path) – The path to the file containing the necessary worker functions: spawn_worker and start_worker. Optionally, the file can also contain a check_function and an init_callback_fn.

  • total_roi (Roi) – The ROI to process.

  • read_roi (Roi) – The ROI to read from for a block.

  • write_roi (Roi) – The ROI to write to for a block.

  • num_workers (int) – The number of workers to use.

  • max_retries (int) – The maximum number of times a task will be retried if failed (either due to failed post check or application crashes or network failure)

  • *args – Additional positional arguments to pass to worker_function.

  • **kwargs – Additional keyword arguments to pass to worker_function.

Returns:

Bool.

Examples

>>> run_blockwise(worker_file, total_roi, read_roi, write_roi, num_workers, max_retries, timeout, upstream_tasks)
dacapo.blockwise.segment_blockwise(segment_function_file: str | upath.UPath, context: funlib.geometry.Coordinate, total_roi: funlib.geometry.Roi, read_roi: funlib.geometry.Roi, write_roi: funlib.geometry.Roi, num_workers: int = 16, max_retries: int = 2, timeout=None, upstream_tasks=None, keep_tmpdir=False, *args, **kwargs)

Run a segmentation function in parallel over a large volume.

Parameters:

  • segment_function_file (str or Path) – The path to the file containing the necessary worker functions: spawn_worker and start_worker. Optionally, the file can also contain a check_function and an init_callback_fn.

  • context (Coordinate) – The context to add to the read and write ROI.

  • total_roi (Roi) – The ROI to process.

  • read_roi (Roi) – The ROI to read from for a block.

  • write_roi (Roi) – The ROI to write to for a block.

  • num_workers (int) – The number of workers to use.

  • max_retries (int) – The maximum number of times a task will be retried if failed (either due to failed post check or application crashes or network failure)

  • timeout (int) – The maximum time in seconds to wait for a worker to complete a task.

  • upstream_tasks (List) – List of upstream tasks.

  • keep_tmpdir (bool) – Whether to keep the temporary directory.

  • *args – Additional positional arguments to pass to worker_function.

  • **kwargs – Additional keyword arguments to pass to worker_function.

Returns:

Bool.

Examples

>>> segment_blockwise(segment_function_file, context, total_roi, read_roi, write_roi, num_workers, max_retries, timeout, upstream_tasks)