Implement dataset.query_objects method#402
Conversation
gatli
left a comment
gatli
left a comment
There was a problem hiding this comment.
Looking good except some Nucleus specific logic 🙂
You might have inferred a wrong class based on a mapping based match ...
| and ground_truth_annotation_label is None | ||
| ): | ||
| confusion_category = ConfusionCategory.TRUE_NEGATIVE | ||
| elif model_prediction_label == ground_truth_annotation_label: |
There was a problem hiding this comment.
This is wrong as it doesn't take into account allowed label mapping. We can have a true positive according to this mapping even though the labels don't match.
The true_positive field indicates if the match was a match.
| confusion_category = infer_confusion_category( | ||
| ground_truth_annotation_label=ground_truth_annotation_label, | ||
| model_prediction_label=model_prediction_label, | ||
| ) |
There was a problem hiding this comment.
Maybe this was a bit of a confusing ask: we should propogate the true_positive to the SDK but then use that to map the confusion type. 🙂
Co-authored-by: Gunnar Atli Thoroddsen <gunnar.thoroddsen@scale.com>
gatli
left a comment
gatli
left a comment
There was a problem hiding this comment.
LGTM 👍
I added a docstring for EvaluationMatch, LMK if there is anything unclear there.
- fixed IOU type
int->float
Nullable has the lame type annotation Optional. You'll get used to it 😉
Thanks! Yeah, originally I wrote |
2 participants
No description provided.