Metrics

We provide a few common metrics for object detection

Metrics for object detection tasks.

mira.metrics.classification_metrics(true_collection, pred_collection)[source]

Compute precision/recall/f1 for each class.

mira.metrics.crop_error_examples(true_collection, pred_collection, threshold=0.3, iou_threshold=0.1)[source]

Get crops of true positives, false negatives, and false positives.

Parameters
  • true_collection (SceneCollection) – A collection of the ground truth scenes.

  • pred_collection (SceneCollection) – A collection of the predicted scenes.

  • threshold – The score threshold for selecting annotations from predicted scenes.

  • iou_threhsold – The IoU threshold for counting a box as a true positive.

Return type

List[Dict[str, List[Annotation]]]

Returns

A list of dicts with “tps”, “fps”, and “fns” with the same length of the input collections. The values in each dict are crops from the original image.

mira.metrics.mAP(true_collection, pred_collection, iou_threshold=0.5)[source]

Compute mAP (mean average precision) for a pair of scene collections.

Parameters
  • true_collection (SceneCollection) – The true scene collection

  • pred_collection (SceneCollection) – The predicted scene collection

  • iou_threshold (float) – The threshold for detection

Return type

Dict[str, float]

Returns

mAP class scores

mira.metrics.precision_recall_curve(true_collection, pred_collection, iou_threshold=0.5)[source]

Compute the precision-recall curve for each of the classes.

Parameters
  • true_collection (SceneCollection) – The true scene collection

  • pred_collection (SceneCollection) – The predicted scene collection

  • iou_threshold (float) – The threshold for detection

Return type

Dict[str, ndarray]

Returns

A dict with category names as keys and array of shape (Ni, 3) which is the precision, recall, and score for each of the predicted boxes for the category.