`BinaryAccuracy` (and similar metrics) incorrectly assumes logits between [0,1] are confidences/probabilities

[golmschenk]

🐛 Bug

This is half-way between a bug and feature request, as the code runs as expected, but produces the incorrect results in some cases. Please update the issue labels if appropriate.

Currently, when using BinaryAccuracy (and other metrics), the metric will be in logits mode if the preds extend beyond the [0,1] range and will apply a sigmoid or similar function. If the preds is within the [0,1] range, it is assumed it is already a confidence/probability value. However, it is possible for the logits to fall within the [0,1] range, and be incorrectly processed as confidences/probabilities. Although in many cases this is rare, training cases with small batch sizes, particularly early in training depending on the initialization, can encounter this issue.

This is notable in cases such as the MulticlassAUROC metric, where a fused softmax is important for numerical precision.

While this is not likely to cause a major differences in training processes, for correctness and reproducibility, this causes an issue.

Potential solution

Allow a way to explicitly state that logits should be assumed (even if the range is in [0,1]).

To Reproduce

Pass [0,1] logits to any of the metrics that assume [0,1] is not logits.