Weighted MCC: A Robust Measure of Multiclass Classifier Performance for Observations with Individual Weights
Published:Dec 25, 2025 05:00
•2 min read
•ArXiv Stats ML
Analysis
This paper introduces a weighted version of the Matthews Correlation Coefficient (MCC) designed to evaluate multiclass classifiers when individual observations have varying weights. The key innovation is the weighted MCC's sensitivity to these weights, allowing it to differentiate classifiers that perform well on highly weighted observations from those with similar overall performance but better performance on lowly weighted observations. The paper also provides a theoretical analysis demonstrating the robustness of the weighted measures to small changes in the weights. This research addresses a significant gap in existing performance measures, which often fail to account for the importance of individual observations. The proposed method could be particularly useful in applications where certain data points are more critical than others, such as in medical diagnosis or fraud detection.
Key Takeaways
- •Introduces a weighted MCC for multiclass classification with individual observation weights.
- •Weighted MCC is sensitive to the weights, prioritizing performance on highly weighted observations.
- •The weighted measures are proven to be robust with respect to small changes in weights.
Reference
“The weighted MCC values are higher for classifiers that perform better on highly weighted observations, and hence is able to distinguish them from classifiers that have a similar overall performance and ones that perform better on the lowly weighted observations.”