Metrics

We provide a few common metrics for object detection

Metrics for object detection tasks.

mira.metrics.classification_metrics(true_collection, pred_collection)

Compute precision/recall/f1 for each class.

mira.metrics.crop_error_examples(true_collection, pred_collection, threshold=0.3, iou_threshold=0.1)

Get crops of true positives, false negatives, and false positives.

Parameters

• true_collection (SceneCollection) – A collection of the ground truth scenes.

• pred_collection (SceneCollection) – A collection of the predicted scenes.

• threshold – The score threshold for selecting annotations from predicted scenes.

• iou_threhsold – The IoU threshold for counting a box as a true positive.

Return type

List[Dict[str, List[Annotation]]]

Returns

A list of dicts with “tps”, “fps”, and “fns” with the same length of the input collections. The values in each dict are crops from the original image.

mira.metrics.mAP(true_collection, pred_collection, iou_threshold=0.5)

Compute mAP (mean average precision) for a pair of scene collections.

Parameters

• true_collection (SceneCollection) – The true scene collection

• pred_collection (SceneCollection) – The predicted scene collection

• iou_threshold (float) – The threshold for detection

Return type

Dict[str, float]

Returns

mAP class scores

mira.metrics.mIOU(true_collection, pred_collection, threshold=0.5)

Compute mIOU for two scene collections

Return type

Dict[str, float]

mira.metrics.precision_recall_curve(true_collection, pred_collection, iou_threshold=0.5)

Compute the precision-recall curve for each of the classes.

Parameters

• true_collection (SceneCollection) – The true scene collection

• pred_collection (SceneCollection) – The predicted scene collection

• iou_threshold (float) – The threshold for detection

Return type

Dict[str, ndarray]

Returns

A dict with category names as keys and array of shape (Ni, 3) which is the precision, recall, and score for each of the predicted boxes for the category.