New Journal Publication: Evaluating AI Without Labels

How do we evaluate anomaly detection models when we don’t have labels? This challenge is at the core of a newly published study by Jiyan Salim Mahmud, Zakarya Farou, and Imre Lendák from our department.

Published in Complex & Intelligent Systems, the paper introduces ASOI (Anomaly Separation and Overlap Index) - a new internal evaluation metric designed specifically for unsupervised anomaly detection.

Measuring Quality Directly from Data: Unlike traditional evaluation approaches that rely on known "ground-truth" labels, ASOI provides a principled way to estimate model quality directly from the data itself. The proposed method improves upon existing techniques by accounting for the overlap between normal and anomalous samples, making the metric more robust in real-world settings where boundaries are often unclear.

Experimental results across multiple datasets show that ASOI correlates more strongly with external performance measures (such as F1-score) compared to commonly used internal metrics.

The department congratulates the authors on this contribution to reliable, label-free evaluation in machine learning.

The article can be found here.