feat: Introduce the PixelLayoutEvaluator to produce confusion matrices for the multi-label layout analysis

[nikos-livathinos]

TL;DR

• Produce confusion matrices and derivatives (precision, recall, f1 matrices and vectors) for the multi-label layout analysis.
• Evaluate at the pixel level.
• The method is applicable even if the layout model does not produce scores.
• In case the predicted classes differ from the ones in the ground truth, it is still possible to produce block-matrices for confusion and derivatives.
• Generate excel reports with visualizations that allow to inspect what kind of prediction mistakes have been done.
• Fast, fully-vectorized implementation.

Detailed description

The PixelLayoutEvaluator computes confusion matrices for the multi-label layout analysis task at the pixel level.
The produced confusion matrices contain fractional counts of pixels, where values on the main diagonal are correct predictions (gains) and values outside of the main diagonal are mis-predictions (penalties). An additional class for the background has been added. Precision, recall and F1 matrices can be easily computed out of the confusion matrix and finally we can extract vectors with precision, recall, f1 values per class. Such matrices/vectors can be computed at each level (page, document, benchmark).

The generated matrices are visualized as colorized excel reports that allow to easily investigate the performance of the layout model and answer questions like:

• Which classes are correctly predicted and to which extend.
• For the mispredictions: Which classes have been predicted instead of the correct ones and to which extend.
• How the predicted classes overlap with each other.
• How well is background predicted.
• etc.

Given that the method applies at a pixel level it is important to have a fast implementation. This is a fully vectorized implementation on top of numpy.

The method is inspired by the paper: https://csitcp.org/paper/10/108csit01.pdf

[github-actions]

✅ DCO Check Passed

Thanks @nikos-livathinos, all your commits are properly signed off. 🎉

[mergify]

Merge Protections

Your pull request matches the following merge protections and will not be merged until they are valid.

🔴 Require two reviewer for test updates

This rule is failing.

When test data is updated, we require two reviewers

• #approved-reviews-by >= 2

🟢 Enforce conventional commit

Wonderful, this rule succeeded.

Make sure that we follow https://www.conventionalcommits.org/en/v1.0.0/

• title ~= ^(fix|feat|docs|style|refactor|perf|test|build|ci|chore|revert)(?:\(.+\))?(!)?: