dacapo.experiments.tasks.evaluators.dummy_evaluator

Classes

DummyEvaluator

A class representing a dummy evaluator. This evaluator is used for testing purposes.

Module Contents

class dacapo.experiments.tasks.evaluators.dummy_evaluator.DummyEvaluator

A class representing a dummy evaluator. This evaluator is used for testing purposes.

criteria

List[str] the evaluation criteria

evaluate(output_array_identifier, evaluation_dataset)

Evaluate the output array against the evaluation dataset.

score()

Return the evaluation scores.

Note

The DummyEvaluator class is used to evaluate the performance of a dummy task.

criteria = ['frizz_level', 'blipp_score']

A list of all criteria for which a model might be “best”. i.e. your criteria might be “precision”, “recall”, and “jaccard”. It is unlikely that the best iteration/post processing parameters will be the same for all 3 of these criteria

Returns:

List[str]

the evaluation criteria

Raises:

NotImplementedError – if the function is not implemented

Examples

>>> evaluator = Evaluator()
>>> evaluator.criteria
[]

Note

This function is used to return the evaluation criteria.

evaluate(output_array_identifier, evaluation_dataset)

Evaluate the given output array and dataset and returns the scores based on predefined criteria.

Parameters:

• output_array_identifier – The output array to be evaluated.

• evaluation_dataset – The dataset to be used for evaluation.

Returns:

An object of DummyEvaluationScores class, with the evaluation scores.

Return type:

DummyEvaluationScore

Raises:

ValueError – if the output array identifier is not valid

Examples

>>> dummy_evaluator = DummyEvaluator()
>>> output_array_identifier = "output_array"
>>> evaluation_dataset = "evaluation_dataset"
>>> dummy_evaluator.evaluate(output_array_identifier, evaluation_dataset)
DummyEvaluationScores(frizz_level=0.0, blipp_score=0.0)

Note

This function is used to evaluate the output array against the evaluation dataset.

property score: dacapo.experiments.tasks.evaluators.dummy_evaluation_scores.DummyEvaluationScores

Return the evaluation scores.

Returns:

An object of DummyEvaluationScores class, with the evaluation scores.

Return type:

DummyEvaluationScores

Examples

>>> dummy_evaluator = DummyEvaluator()
>>> dummy_evaluator.score
DummyEvaluationScores(frizz_level=0.0, blipp_score=0.0)

Note

This function is used to return the evaluation scores.