MAIBAI

Developing a metrological framework for assessment of image-based Artificial Intelligence systems for disease detection

Image-based AI systems for disease detection are increasingly being developed, making necessary their effectiveness and trustworthiness in heterogeneous clinical settings, as well as their evaluation by approved guidelines. To address these points, MAIBAI aims at developing a standardised and impartial framework for performance, generalisability and suitability assessment of AI tools, to enable a more efficient, reliable and reproducible validation of image-based AI systems for disease detection. Using breast screening as an exemplar, AI tools will be benchmarked on a large real-world database of mammographic images, with the final goal of designing a metrological framework for AI assessment and explainability in diagnostic imaging.

The exponential increase in healthcare data over the last decade, as well as the fast-paced technology developments, have resulted in promising novel AI approaches for diagnostic applications and risk prediction. However, the adoption of AI in clinical settings remains limited, mostly due to i) limited data quality and interoperability across heterogeneous clinical centres and electronic health records, ii) absence of robust validation procedures, iii) distrust of predictions and decisions generated by AI systems, and iv) lack of harmonised government proposals and consensus guidelines on steps for their adoption.

To enable the implementation of image-based AI systems for disease detection, MAIBAI addresses the following specific needs:

• Test of AI tools on large and high-quality medical imaging databases, with data categorised and integrated based on clinically relevant subgroups and image acquisition key factors;
• Provision of a clear methodology to benchmark the quality of predictive AI models, with relevant associated metrics, and interpretation methods for explainable and traceable AI tools;
• Design of a global, standardised, and impartial AI assessment framework.

Project funding:

The project 22HLT05 MAIBAI, started in September 2023, has received funding from the European Partnership on Metrology, co-financed from the European Union’s Horizon Europe Research and Innovation Programme and by the Participating States. Funding for the UK partners is provided by Innovate UK under the Horizon Europe Guarantee Extension.

Funded by the European Union. Views and opinions expressed are however those of the author(s) only and do not necessarily reflect those of the European Union or EURAMET. Neither the European Union nor the granting authority can be held responsible for them.